Running through another Elasticsearch deploy this weekend, I thought I would jot it down so others can see/use it too.
| Component | Github | Charmstore |
|---|---|---|
| elasticsearch | layer-elasticsearch | cs:~omnivector/elasticsearch |
| layer-elastic-base | ||
| kibana | layer-kibana | cs:~omnivector/kibana |
| layer-elasticsearch-client | ||
| layer-elastic-base |
This walkthrough uses Juju to deploy a heterogeneous Elasticsearch cluster consisting of:
* 18 Data Nodes
* 3 Master Nodes
* 1 Ingest Node
Note: My ingest use case is minimal so I only deploy a single ingest node, but could have deployed many more if needed.
Deploy and Relate the Elasticsearch Components
# Master Nodes
juju deploy cs:~omnivector/elasticsearch es-master -n3 \
--constraints "instance-type=t3.2xlarge root-disk=20G" \
--config java-opts="-Xms16g -Xmx16g" --config cluster-name="es-bdx-testing" \
--config node-type=master
# Data Nodes
juju deploy cs:~omnivector/elasticsearch es-data -n18 \
--constraints "instance-type=t3.2xlarge root-disk=20G" \
--config java-opts="-Xms16g -Xmx16g" --config cluster-name="es-bdx-testing" \
--storage data=ebs-ssd,500G --config node-type=data
# Ingest Node(s)
juju deploy cs:~omnivector/elasticsearch es-ingest -n1 \
--constraints "instance-type=t3.2xlarge root-disk=20G" \
--config java-opts="-Xms16g -Xmx16g" --config cluster-name="es-bdx-testing" \
--config node-type=ingest
# Relate
juju relate es-master:provide-master es-data:require-master
juju relate es-master:provide-master es-ingest:require-master
At this point you should have an Elasticsearch cluster running with 3 master, 18 data, and 1 ingest node.
Deploy and Relate Kibana
juju deploy cs:~omnivector/kibana \
--constraints "instance-type=t3.medium root-disk=20G"
juju relate kibana:elasticsearch es-master:client
Final Result
Model Controller Cloud/Region Version SLA Timestamp
my-bdx-es-dev-00 pdl-aws aws/us-west-2 2.5.4 unsupported 14:44:17Z
App Version Status Scale Charm Store Rev OS Notes
es-data 6.7.2 active 18 elasticsearch jujucharms 37 ubuntu exposed
es-ingest 6.7.2 active 1 elasticsearch jujucharms 37 ubuntu
es-master 6.7.2 active 3 elasticsearch jujucharms 37 ubuntu exposed
kibana 6.7.2 active 1 kibana jujucharms 7 ubuntu exposed
Unit Workload Agent Machine Public address Ports Message
es-data/27 active idle 37 172.31.104.38 9200/tcp,9300/tcp Elasticsearch Running - 18 x data nodes
es-data/28 active idle 38 172.31.102.221 9200/tcp,9300/tcp Elasticsearch Running - 18 x data nodes
es-data/29 active idle 39 172.31.102.184 9200/tcp,9300/tcp Elasticsearch Running - 18 x data nodes
es-data/30 active idle 40 172.31.104.57 9200/tcp,9300/tcp Elasticsearch Running - 18 x data nodes
es-data/31 active idle 41 172.31.102.130 9200/tcp,9300/tcp Elasticsearch Running - 18 x data nodes
es-data/32 active idle 42 172.31.102.220 9200/tcp,9300/tcp Elasticsearch Running - 18 x data nodes
es-data/33 active idle 43 172.31.102.107 9200/tcp,9300/tcp Elasticsearch Running - 18 x data nodes
es-data/34* active idle 44 172.31.104.104 9200/tcp,9300/tcp Elasticsearch Running - 18 x data nodes
es-data/35 active idle 45 172.31.104.162 9200/tcp,9300/tcp Elasticsearch Running - 18 x data nodes
es-data/36 active idle 50 172.31.104.13 9200/tcp,9300/tcp Elasticsearch Running - 18 x data nodes
es-data/37 active idle 51 172.31.102.153 9200/tcp,9300/tcp Elasticsearch Running - 18 x data nodes
es-data/38 active idle 52 172.31.104.8 9200/tcp,9300/tcp Elasticsearch Running - 18 x data nodes
es-data/39 active idle 53 172.31.103.59 9200/tcp,9300/tcp Elasticsearch Running - 18 x data nodes
es-data/40 active idle 54 172.31.104.166 9200/tcp,9300/tcp Elasticsearch Running - 18 x data nodes
es-data/41 active idle 55 172.31.102.30 9200/tcp,9300/tcp Elasticsearch Running - 18 x data nodes
es-data/42 active idle 56 172.31.102.132 9200/tcp,9300/tcp Elasticsearch Running - 18 x data nodes
es-data/43 active idle 57 172.31.103.155 9200/tcp,9300/tcp Elasticsearch Running - 18 x data nodes
es-data/44 active idle 58 172.31.103.112 9200/tcp,9300/tcp Elasticsearch Running - 18 x data nodes
es-ingest/0* active idle 47 172.31.103.119 9200/tcp,9300/tcp Elasticsearch Running - 1 x ingest nodes
es-master/6* active idle 25 172.31.102.234 9200/tcp,9300/tcp Elasticsearch Running - 3 x master nodes
es-master/7 active idle 26 172.31.102.93 9200/tcp,9300/tcp Elasticsearch Running - 3 x master nodes
es-master/8 active idle 27 172.31.103.64 9200/tcp,9300/tcp Elasticsearch Running - 3 x master nodes
kibana/3 active idle 49 172.31.103.57 80/tcp Kibana available
Machine State DNS Inst id Series AZ Message
25 started 172.31.102.234 i-0f2d5edc6002af708 bionic us-west-2a running
26 started 172.31.102.93 i-0cb4d854d21dec9f3 bionic us-west-2a running
27 started 172.31.103.64 i-0b5678c06c6f4cab7 bionic us-west-2b running
37 started 172.31.104.38 i-0b772836dc61da756 bionic us-west-2c running
38 started 172.31.102.221 i-0fa0be0eef3b64d00 bionic us-west-2a running
39 started 172.31.102.184 i-01b8d99af0635b58a bionic us-west-2a running
40 started 172.31.104.57 i-0ffd933a2af314de2 bionic us-west-2c running
41 started 172.31.102.130 i-09ece2c98bc7ea034 bionic us-west-2a running
42 started 172.31.102.220 i-0423e0ea5281eebca bionic us-west-2a running
43 started 172.31.102.107 i-09c6acefdc0d1d89b bionic us-west-2a running
44 started 172.31.104.104 i-09753d46515d30947 bionic us-west-2c running
45 started 172.31.104.162 i-0fcdfce2d39951edd bionic us-west-2c running
47 started 172.31.103.119 i-05f46dfd3a29933d7 bionic us-west-2b running
49 started 172.31.103.57 i-02c5f015a73ad5bfe bionic us-west-2b running
50 started 172.31.104.13 i-0f839ddbc728ea04c bionic us-west-2c running
51 started 172.31.102.153 i-020b38070c87de9b8 bionic us-west-2a running
52 started 172.31.104.8 i-065f61eae901ef600 bionic us-west-2c running
53 started 172.31.103.59 i-0ea9dfe9fd0075c66 bionic us-west-2b running
54 started 172.31.104.166 i-08ca7dfb1a1460c53 bionic us-west-2c running
55 started 172.31.102.30 i-0b6f2eeecf4806135 bionic us-west-2a running
56 started 172.31.102.132 i-0670349e0cd3695ae bionic us-west-2a running
57 started 172.31.103.155 i-0e025ba96be105655 bionic us-west-2b running
58 started 172.31.103.112 i-07e1412f4619bf527 bionic us-west-2b running
As always, pull requests welcome.
Enjoy!
