SoFunction
Updated on 2025-04-10

Redis advanced error-tolerant switching implementation

When a host master crashes, does it replace the host by its slave?

Check cluster status

127.0.0.1:6385> cluster nodes
c8ff33e8da5fd8ef821c65974dda304d2e3327f9 192.168.58.129:6382@16382 slave f6b1fd5e58df90782f602b484c2011d52fc3482d 0 1733220836918 1 connected
6656947825a5aed22e4ce1818c1e743739219374 192.168.58.212:6386@16386 slave 437c0ffe1015fc8c7049cfeb0af05fee17e4b0a1 0 1733220835000 5 connected
73b5567a02ee9f958f00324bf25bee5ce8eb23bc 192.168.58.130:6384@16384 slave e1dafb251016aeaa59edf64c4f7bb3d4639f560a 0 1733220834890 3 connected
f6b1fd5e58df90782f602b484c2011d52fc3482d 192.168.58.212:6385@16385 myself,master - 0 0 1 connected 0-5460
437c0ffe1015fc8c7049cfeb0af05fee17e4b0a1 192.168.58.130:6383@16383 master - 0 1733220835395 5 connected 10923-16383
e1dafb251016aeaa59edf64c4f7bb3d4639f560a 192.168.58.129:6381@16381 master - 0 1733220836410 3 connected 5461-10922

Stop host 6385

127.0.0.1:6385> SHUTDOWN

(10.02s)
not connected> 
not connected> 
not connected> 
not connected> 

Use the redis-cli command to enter redis to view the cluster status again

127.0.0.1:6382> info replication
# Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:f292ca63e04db6e7717cc5067f408bf733963bb2
master_replid2:6ab6cdfeb1902aaa71279434c6b2c0e46d28a7f4
master_repl_offset:0
second_repl_offset:1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
127.0.0.1:6382> cluster nodes
437c0ffe1015fc8c7049cfeb0af05fee17e4b0a1 192.168.58.130:6383@16383 master - 0 1733221855390 5 connected 10923-16383
f6b1fd5e58df90782f602b484c2011d52fc3482d 192.168.58.212:6385@16385 master,fail - 1733221087102 1733221084545 1 disconnected
6656947825a5aed22e4ce1818c1e743739219374 192.168.58.212:6386@16386 slave 437c0ffe1015fc8c7049cfeb0af05fee17e4b0a1 0 1733221854869 5 connected
73b5567a02ee9f958f00324bf25bee5ce8eb23bc 192.168.58.130:6384@16384 slave e1dafb251016aeaa59edf64c4f7bb3d4639f560a 0 1733221853318 3 connected
e1dafb251016aeaa59edf64c4f7bb3d4639f560a 192.168.58.129:6381@16381 master - 0 1733221855000 3 connected 5461-10922
c8ff33e8da5fd8ef821c65974dda304d2e3327f9 192.168.58.129:6382@16382 myself,master - 0 1733221854000 7 connected 0-5460

6382 successfully became host

So will the original 6385 host be restored to regain its position?

Won't!

Current cluster status:

127.0.0.1:6382> cluster nodes
437c0ffe1015fc8c7049cfeb0af05fee17e4b0a1 192.168.58.130:6383@16383 master - 0 1733221855390 5 connected 10923-16383
f6b1fd5e58df90782f602b484c2011d52fc3482d 192.168.58.212:6385@16385 master,fail - 1733221087102 1733221084545 1 disconnected
6656947825a5aed22e4ce1818c1e743739219374 192.168.58.212:6386@16386 slave 437c0ffe1015fc8c7049cfeb0af05fee17e4b0a1 0 1733221854869 5 connected
73b5567a02ee9f958f00324bf25bee5ce8eb23bc 192.168.58.130:6384@16384 slave e1dafb251016aeaa59edf64c4f7bb3d4639f560a 0 1733221853318 3 connected
e1dafb251016aeaa59edf64c4f7bb3d4639f560a 192.168.58.129:6381@16381 master - 0 1733221855000 3 connected 5461-10922
c8ff33e8da5fd8ef821c65974dda304d2e3327f9 192.168.58.129:6382@16382 myself,master - 0 1733221854000 7 connected 0-5460

Re-specify the 6385 configuration file to start the service: 

127.0.0.1:6385> cluster nodes
0a820807a70885c1d87d141824d3869dcf2418ee 192.168.58.130:6383@16383 master - 0 1733300452328 3 connected 5461-10922
91cf395e8f04a5df1aeec4602dd38b3e4a708d55 192.168.58.130:6384@16384 slave ba40235fdad71fb36c3bf165ea51055825d453c2 0 1733300451285 1 connected
be191621d4aa881d864a113c385d58b751753392 192.168.58.212:6385@16385 myself,slave fedf4b13be2122f44dde749a08c4b943582b3566 0 0 7 connected
30e5bdec200a671e4de666551d4489f0035235ba 192.168.58.212:6386@16386 slave 0a820807a70885c1d87d141824d3869dcf2418ee 0 1733300452892 3 connected
ba40235fdad71fb36c3bf165ea51055825d453c2 192.168.58.129:6381@16381 master - 0 1733300453572 1 connected 0-5460
fedf4b13be2122f44dde749a08c4b943582b3566 192.168.58.129:6382@16382 master - 0 1733300452659 7 connected 10923-16383

Node master-slave adjustment

The current cluster state is 6385, and 6382 is the main one.

127.0.0.1:6385> cluster nodes
0a820807a70885c1d87d141824d3869dcf2418ee 192.168.58.130:6383@16383 master - 0 1733301081028 3 connected 5461-10922
91cf395e8f04a5df1aeec4602dd38b3e4a708d55 192.168.58.130:6384@16384 slave ba40235fdad71fb36c3bf165ea51055825d453c2 0 1733301082183 1 connected
be191621d4aa881d864a113c385d58b751753392 192.168.58.212:6385@16385 myself,slave fedf4b13be2122f44dde749a08c4b943582b3566 0 0 7 connected
30e5bdec200a671e4de666551d4489f0035235ba 192.168.58.212:6386@16386 slave 0a820807a70885c1d87d141824d3869dcf2418ee 0 1733301082954 3 connected
ba40235fdad71fb36c3bf165ea51055825d453c2 192.168.58.129:6381@16381 master - 0 1733301082057 1 connected 0-5460
fedf4b13be2122f44dde749a08c4b943582b3566 192.168.58.129:6382@16382 master - 0 1733301081260 7 connected 10923-16383

Execute the commandcluster favlover, master-slave node adjustment, 6385 is the main and 6382 is the slave

127.0.0.1:6385> cluster nodes
0a820807a70885c1d87d141824d3869dcf2418ee 192.168.58.130:6383@16383 master - 0 1733301262000 3 connected 5461-10922
91cf395e8f04a5df1aeec4602dd38b3e4a708d55 192.168.58.130:6384@16384 slave ba40235fdad71fb36c3bf165ea51055825d453c2 0 1733301261585 1 connected
be191621d4aa881d864a113c385d58b751753392 192.168.58.212:6385@16385 myself,master - 0 0 8 connected 10923-16383
30e5bdec200a671e4de666551d4489f0035235ba 192.168.58.212:6386@16386 slave 0a820807a70885c1d87d141824d3869dcf2418ee 0 1733301262869 3 connected
ba40235fdad71fb36c3bf165ea51055825d453c2 192.168.58.129:6381@16381 master - 0 1733301263074 1 connected 0-5460
fedf4b13be2122f44dde749a08c4b943582b3566 192.168.58.129:6382@16382 slave be191621d4aa881d864a113c385d58b751753392 0 1733301261700 8 connected

Command supplement

When the master node fails, the slave node does not become a host and uses the following command to force it to be promoted to the host

redis-cli -p 6382 cluster failover force

from6382The node executes the command to rejoin the cluster

redis-cli -a 111111 -p 6382 cluster meet 192.168.58.212 6385

This is the end of this article about the implementation of Redis's advanced fault-tolerant switching. For more related Redis fault-tolerant switching, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!