搭建redis集群

创建redis cluster node

Redis cluster 最小配置项

1
2
3
4
5
port 7000
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

参数说明

  • cluster-enabled <yes/no>: If yes enables Redis Cluster support in a specific Redis instance. Otherwise the instance starts as a stand alone instance as usually.
  • cluster-config-file <filename>: Note that despite the name of this option, this is not an user editable configuration file, but the file where a Redis Cluster node automatically persists the cluster configuration (the state, basically) every time there is a change, in order to be able to re-read it at startup. The file lists things like the other nodes in the cluster, their state, persistent variables, and so forth. Often this file is rewritten and flushed on disk as a result of some message reception.
  • cluster-node-timeout <milliseconds>: The maximum amount of time a Redis Cluster node can be unavailable, without it being considered as failing. If a master node is not reachable for more than the specified amount of time, it will be failed over by its slaves. This parameter controls other important things in Redis Cluster. Notably, every node that can’t reach the majority of master nodes for the specified amount of time, will stop accepting queries.
  • cluster-slave-validity-factor <factor>: If set to zero, a slave will always try to failover a master, regardless of the amount of time the link between the master and the slave remained disconnected. If the value is positive, a maximum disconnection time is calculated as the node timeout value multiplied by the factor provided with this option, and if the node is a slave, it will not try to start a failover if the master link was disconnected for more than the specified amount of time. For example if the node timeout is set to 5 seconds, and the validity factor is set to 10, a slave disconnected from the master for more than 50 seconds will not try to failover its master. Note that any value different than zero may result in Redis Cluster to be unavailable after a master failure if there is no slave able to failover it. In that case the cluster will return back available only when the original master rejoins the cluster.
  • cluster-migration-barrier <count>: Minimum number of slaves a master will remain connected with, for another slave to migrate to a master which is no longer covered by any slave. See the appropriate section about replica migration in this tutorial for more information.
  • cluster-require-full-coverage <yes/no>: If this is set to yes, as it is by default, the cluster stops accepting writes if some percentage of the key space is not covered by any node. If the option is set to no, the cluster will still serve queries even if only requests about a subset of keys can be processed.

创建实例

创建六个redis node实例,包括三个master,三个slave

1
2
3
mkdir cluster-test
cd cluster-test
mkdir 7000 7001 7002 7003 7004 7005
1
2
cd 7000
../redis-server ./redis.conf

每个节点会生成node.conf的配置文件,生成唯一的Node ID

a0f384e70751114a8b041bd5951b6ccd07e1fcb2 :0 myself,master - 0 0 0 connected
vars currentEpoch 0 lastVoteEpoch 0

创建cluster

通过使用 redis-trib.rb 创建

1
/usr/local/redis-3.2.3/src/redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
127.0.0.1:7000
127.0.0.1:7001
127.0.0.1:7002
Adding replica 127.0.0.1:7003 to 127.0.0.1:7000
Adding replica 127.0.0.1:7004 to 127.0.0.1:7001
Adding replica 127.0.0.1:7005 to 127.0.0.1:7002
M: 2448473eab43c49e4a262f77f0ed94d324ed813c 127.0.0.1:7000
slots:0-5460 (5461 slots) master
M: 35d9757eff0e6dce89aa46d9b71186329ebcd679 127.0.0.1:7001
slots:5461-10922 (5462 slots) master
M: 634799c3b2b24c054010298158972e430a95cd2a 127.0.0.1:7002
slots:10923-16383 (5461 slots) master
S: 2a14cd7b9f6417dc511173c5c9051545be1f847b 127.0.0.1:7003
replicates 2448473eab43c49e4a262f77f0ed94d324ed813c
S: 5aab7783b72c4bcf70731d8243f2556be6ab8c57 127.0.0.1:7004
replicates 35d9757eff0e6dce89aa46d9b71186329ebcd679
S: a0f384e70751114a8b041bd5951b6ccd07e1fcb2 127.0.0.1:7005
replicates 634799c3b2b24c054010298158972e430a95cd2a

Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join..
>>> Performing Cluster Check (using node 127.0.0.1:7000)
M: 2448473eab43c49e4a262f77f0ed94d324ed813c 127.0.0.1:7000
slots:0-5460 (5461 slots) master
M: 35d9757eff0e6dce89aa46d9b71186329ebcd679 127.0.0.1:7001
slots:5461-10922 (5462 slots) master
M: 634799c3b2b24c054010298158972e430a95cd2a 127.0.0.1:7002
slots:10923-16383 (5461 slots) master
M: 2a14cd7b9f6417dc511173c5c9051545be1f847b 127.0.0.1:7003
slots: (0 slots) master
replicates 2448473eab43c49e4a262f77f0ed94d324ed813c
M: 5aab7783b72c4bcf70731d8243f2556be6ab8c57 127.0.0.1:7004
slots: (0 slots) master
replicates 35d9757eff0e6dce89aa46d9b71186329ebcd679
M: a0f384e70751114a8b041bd5951b6ccd07e1fcb2 127.0.0.1:7005
slots: (0 slots) master
replicates 634799c3b2b24c054010298158972e430a95cd2a
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

这样通过redis-trib.rb,我们就直接将六个节点关联成一个拥有3个master,3个slave的集群了。

端口为7000的redis实例的node.conf就变为

1
2
3
4
5
6
a0f384e70751114a8b041bd5951b6ccd07e1fcb2 127.0.0.1:7005 slave 634799c3b2b24c054010298158972e430a95cd2a 0 1470382848905 6 connected
5aab7783b72c4bcf70731d8243f2556be6ab8c57 127.0.0.1:7004 slave 35d9757eff0e6dce89aa46d9b71186329ebcd679 0 1470382848401 5 connected
2a14cd7b9f6417dc511173c5c9051545be1f847b 127.0.0.1:7003 slave 2448473eab43c49e4a262f77f0ed94d324ed813c 0 1470382848905 4 connected
35d9757eff0e6dce89aa46d9b71186329ebcd679 127.0.0.1:7001 master - 0 1470382848905 2 connected 5461-10922
634799c3b2b24c054010298158972e430a95cd2a 127.0.0.1:7002 master - 0 1470382849421 3 connected 10923-16383
2448473eab43c49e4a262f77f0ed94d324ed813c 127.0.0.1:7000 myself,master - 0 0 1 connected 0-5460

查看cluster状态

1
2
3
4
5
6
7
8
9
10
11
12
127.0.0.1:7000> CLUSTER INFO
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_sent:4326
cluster_stats_messages_received:4326

测试master node挂掉后

1
2
3
4
5
6
7
8
9
10
11
12
13
[root@local tmp]# redis-cli -p 7000 cluster nodes | grep master
35d9757eff0e6dce89aa46d9b71186329ebcd679 127.0.0.1:7001 master - 0 1470383992263 2 connected 5461-10922
634799c3b2b24c054010298158972e430a95cd2a 127.0.0.1:7002 master - 0 1470383990234 3 connected 10923-16383
2448473eab43c49e4a262f77f0ed94d324ed813c 127.0.0.1:7000 myself,master - 0 0 1 connected 0-5460

[root@local tmp]# redis-cli -p 7002 debug segfault
Error: Server closed the connection

[root@local tmp]# redis-cli -p 7000 cluster nodes | grep master
a0f384e70751114a8b041bd5951b6ccd07e1fcb2 127.0.0.1:7005 master - 0 1470384268524 8 connected 10923-16383
35d9757eff0e6dce89aa46d9b71186329ebcd679 127.0.0.1:7001 master - 0 1470384270546 2 connected 5461-10922
634799c3b2b24c054010298158972e430a95cd2a 127.0.0.1:7002 master,fail - 1470384025489 1470384023169 3 disconnected
2448473eab43c49e4a262f77f0ed94d324ed813c 127.0.0.1:7000 myself,master - 0 0 1 connected 0-5460

可以看到,当master节点7002不可用之后,作为slave节点的7005在集群里会升级为master节点

坚持原创技术分享,您的支持将鼓励我继续创作!