Redis-Cluster
My redis-cluster is ever growing, as such I thought it would be wise to get a working piece of automation around it.
30 Second Overview
The redis charm has gained a new functionality that allows it to be deployed in cluster mode. To run Redis in cluster mode, just set cluster-enabled to true when you deploy the charm. This will tell Redis to configure itself in cluster mode. Some lifecycle automation has been added to the charm to facilitate auto-rebalancing when units are added or removed.
| Component | Github | Charmstore |
|---|---|---|
| redis | layer-redis | cs:~omnivector/redis |
| interface-redis |
juju deploy cs:~omnivector/redis -n9 --config cluster-enabled=true
Gives us
Model Controller Cloud/Region Version SLA Timestamp
bdx-redis-test-00 pdl-aws aws/us-west-2 2.5.4 unsupported 15:54:16Z
App Version Status Scale Charm Store Rev OS Notes
redis 5.0.3 active 9 redis jujucharms 17 ubuntu
Unit Workload Agent Machine Public address Ports Message
redis/15 active idle 15 172.31.102.4 6379/tcp successfully clustered
redis/16 active idle 16 172.31.102.41 6379/tcp successfully clustered
redis/17 active idle 17 172.31.102.121 6379/tcp successfully clustered
redis/18 active idle 18 172.31.102.8 6379/tcp successfully clustered
redis/19 active idle 19 172.31.103.174 6379/tcp successfully clustered
redis/20 active idle 20 172.31.104.59 6379/tcp successfully clustered
redis/21 active idle 21 172.31.104.231 6379/tcp successfully clustered
redis/22 active idle 22 172.31.103.120 6379/tcp successfully clustered
redis/23 active idle 23 172.31.103.145 6379/tcp successfully clustered
Machine State DNS Inst id Series AZ Message
15 started 172.31.102.4 i-02b2a4c978a601800 bionic us-west-2a running
16 started 172.31.102.41 i-063d5b602cde3155c bionic us-west-2a running
17 started 172.31.102.121 i-056926746e9fe85ef bionic us-west-2a running
18 started 172.31.102.8 i-003d79f51b8db8cf2 bionic us-west-2a running
19 started 172.31.103.174 i-02225abf5b11e1a95 bionic us-west-2b running
20 started 172.31.104.59 i-0ea30e9c40a53b9bd bionic us-west-2c running
21 started 172.31.104.231 i-064a3a6f185a1915a bionic us-west-2c running
22 started 172.31.103.120 i-03a0e74ab9d23be87 bionic us-west-2b running
23 started 172.31.103.145 i-02f12d738ec0f9f04 bionic us-west-2b running
We can use the redis-cli to verify the node/slot configuration is as intended (balanced).
juju run --unit redis/15 "/snap/bin/redis-bdx.redis-cli cluster nodes"
ff3f173df80e284af39be70d94fc55ec946031ed 172.31.103.174:6379@16379 master - 0 1556985285031 7 connected 680-819 1365-1484 2731-3120 5462-5851 8193-8582 13654-14043
0b32363760c35c5545ec32c454b0067ce7fc461f 172.31.104.59:6379@16379 master - 0 1556985285834 8 connected 228-295 394-549 820-955 1485-1776 3121-3412 5852-6143 8583-8874 14044-14335
e80fc21cd4191dddfea414083d350ed2ec7c6c21 172.31.104.231:6379@16379 master - 0 1556985285000 4 connected 6372-6826 10923-12287
8d4d4bd45943a9dea0b4e750fc3717575639c5d3 172.31.102.41:6379@16379 master - 0 1556985284000 5 connected 2005-2184 6827-7646 12288-13107
9bdd424266aa237cd539851344da28962bdb45ba 172.31.102.121:6379@16379 master - 0 1556985284829 6 connected 1184-1364 2185-2730 5461 7647-8192 13108-13653
0a932ed5563ac81693156e6d0bf5161ab5a1b941 172.31.102.4:6379@16379 myself,master - 0 1556985283000 3 connected 14564-16383
00fe7e2a9448d9810641bb277c0f57a33c2e150b 172.31.103.120:6379@16379 master - 0 1556985285000 9 connected 0-227 296-393 550-679 956-1183 1777-2004 3413-3640 6144-6371 8875-9102 14336-14563
75b84942a6d8d31bd30d89ec96e23e028fbb1189 172.31.102.8:6379@16379 master - 0 1556985284528 1 connected 3641-5460
534b6b989f3bec8cb841c9ef2a03d52573976a82 172.31.103.145:6379@16379 master - 0 1556985284000 2 connected 9103-10922
Example Rebalancing
This charm will automatically rebalance slots to other nodes in the cluster when a unit is pragmatically added or removed.
Scale Down
Let’s remove a few units and see what happens.
$ juju remove-unit redis/21
removing unit redis/21
$ juju remove-unit redis/22
removing unit redis/22
$ juju remove-unit redis/23
removing unit redis/23
Now run redis-bdx.redis-cli cluster nodes to see that only 6 nodes exist and contain all of the slots that were previously on the nodes we removed.
juju run --unit redis/3 "/snap/bin/redis-bdx.redis-cli cluster nodes"
139e305fab256c675909a1cd5504194ef79b55ed 172.31.104.137:6379@16379 master - 0 1556984153564 27 connected 2005-2184 3804-4193 6827-7646 8059-8192 12288-13201 14629-14921
883eebfe36c773c5a48b40cc4bd2d86879431517 172.31.102.83:6379@16379 master - 0 1556984153063 26 connected 1184-1364 2185-2231 2688-2730 3641-3803 5461 7647-7830 9103-10922 15508-15799
35f0715fb296aaa75efd1486e981738b5fd6437f 172.31.104.4:6379@16379 myself,master - 0 1556984152000 31 connected 228-549 820-955 5364-5460 5462-6143 7831-8058 8193-8872 13202-13429 14564-14628 16092-16383
ea8ec8f18d808648802e407c2c5438d6144289b7 172.31.103.243:6379@16379 master - 0 1556984153564 30 connected 2460-2687 4974-5363 6372-6826 10923-12287 15800-16091
0663529d652411e0292b34d36b521ad90f8d0eef 172.31.102.238:6379@16379 master - 0 1556984152562 28 connected 778-819 1365-1776 2232-2459 2731-3412 4194-4583 8873-8874 13654-14335 14922-15214
d81963c014b2b860640d79fb9fb884a383f5edb1 172.31.103.194:6379@16379 master - 0 1556984154565 29 connected 0-227 550-777 956-1183 1777-2004 3413-3640 4584-4973 6144-6371 8875-9102 13430-13653 14336-14563 15215-15507
As you can see the slots from units 1, 2, and 3 have been rebalanced to the remaining nodes.
Scale Up
Add another few units to our cluster of now 6 nodes (as we removed 3 in the last step).
juju add-unit redis -n 6
After the nodes have settled, go ahead and check your slots/nodes again to see the new node additions have been added to the cluster and the slots rebalanced to the new nodes appropriately.