设为首页收藏本站
开启辅助访问
Redis4.0.10集群研究和实践 您所在的位置：网站首页 › Redis集群重启一个节点 › Redis4.0.10集群研究和实践

Redis4.0.10集群研究和实践

2023-04-22 23:20| 来源: 网络整理| 查看: 265

一、Redis4.0.10集群搭建前准备工作

　　Redis 是我们目前大规模使用的缓存中间件，由于它强大高效而又便捷的功能，得到广泛的使用。　　

　　Redis在2015年发布了3.0.0，官方支持了redis cluster。至此结束了redis没有集群的时代，之前使用的redis cluster 最多的是twitter发布的Twemproxy和豌豆荚开发的codis 。

  一、Redis Cluster主从模式

　　redis cluster 为了保证数据的高可用性，加入了主从模式，一个主节点对应一个或多个从 节点，主节点提供数据存取，从节点则是从主节点拉取数据备份，当这个主节点挂掉后，就会 有这个从节点选取一个来充当主节点，从而保证集群不会挂掉。　　上面例子里, 集群有ABC三个主节点,如果这3个节点都没有加入从节点，如果 B挂掉了，我们就 无法访问整个集群了。A和C的 slot 也无法访问。　　所以在集群建立的时候，一定要为每个主节点都添加了从节点, 比如像这样, 集群包含主节点 A、B、C, 以及从节点A1、B1、C1, 那么即使B挂掉系统也可以继续正确工作。　　B1节点替代了B节点，所以Redis集群将会选择B1节点作为新的主节点，集群将会继续正确地 提供服务。 当B重新开启后，它就会变成B1的从节点

　　1、如果节点B和B1同时挂了，Redis集群就无法继续正常的提供服务了 

　　流程下图所示

　　

Redis4.0.10集群研究和实践_sed

 

　　

Redis4.0.10集群研究和实践_redis_02

  二、环境初始化 　　1、架构如下

　　

Redis4.0.10集群研究和实践_sed_03

　　2、设置主机名hostnamectl set-hostname Redis01hostnamectl set-hostname Redis02hostnamectl set-hostname　　3、做主机映射cat /etc/hosts10.0.0.71 Redis0110.0.0.72 Redis0210.0.0.73 Redis03EOF

　　分发集群其他节点

for i in Redis02 Redis03;do scp /etc/hosts $i:/etc/;done　　4、集群之间免密钥认证#创建证书ssh-keygen #一路回车即可#分发证书，需要输入对应主机的账号和密码for i in Redis01 Redis02 Redis03;do ssh-copy-id $i;done　　5、环境初始化for i in Redis01 Redis02 Redis03;do ssh -n $i "mkdir -p /opt/scripts/shell /opt/src /opt/data/redis-cluster && exit";done

Redis4.0.10集群研究和实践_linux_04

Redis4.0.10集群研究和实践_sed_05

cat>/opt/scripts/shell/init_env.sh /tmp/crontab2.tmpcrontab /tmp/crontab2.tmpsystemctl start ntpdate.service#6.set security limitecho "* soft nofile 65536" >> /etc/security/limits.confecho "* hard nofile 65536" >> /etc/security/limits.confecho "* soft nproc 65536" >> /etc/security/limits.confecho "* hard nproc 65536" >> /etc/security/limits.confecho "* soft memlock unlimited" >> /etc/security/limits.confecho "* hard memlock unlimited" >> /etc/security/limits.confEOF

/opt/scripts/shell/init_env.sh

for i in Redis02 Redis03;do scp /opt/scripts/shell/init_env.sh $i:/opt/scripts/shell/ && ssh -n $i "/bin/bash /opt/scripts/shell/init_env.sh && exit";done二、Redis Cluster集群研究和实践  一、Redis01编译安装Redis4.0.10 　　1、编写安装Redis脚本cat>/opt/scripts/shell/install_redis.sh "Aniuredis123", :db => 0, :driver => nil, :id => nil, :tcp_keepalive => 0, :reconnect_attempts => 1, :inherit_socket => false } 　　3、创建集群

新建集群, 使用create命令. --replicas 1 参数表示为每个主节点创建一个从节点. 其他参数是实例的地址集合。 

Redis-trib会提示你做了什么配置, 输入yes接受. 集群就被配置和加入了, 意思是, 实例会经过互相交流后启动。 

　　若gem安装的redis版本与redis集群版本不同，有可能报错如下（解决办法：将报错行注释掉）

Redis4.0.10集群研究和实践_linux_04

Redis4.0.10集群研究和实践_sed_05

/opt/redis4/bin/redis-trib.rb:885:in `sort!': undefined method `' for :m:Symbol (NoMethodError) from /opt/redis4/bin/redis-trib.rb:885:in `get_anti_affinity_score' from /opt/redis4/bin/redis-trib.rb:883:in `each' from /opt/redis4/bin/redis-trib.rb:883:in `get_anti_affinity_score' from /opt/redis4/bin/redis-trib.rb:867:in `each' from /opt/redis4/bin/redis-trib.rb:867:in `get_anti_affinity_score' from /opt/redis4/bin/redis-trib.rb:782:in `optimize_anti_affinity' from /opt/redis4/bin/redis-trib.rb:778:in `alloc_slots' from /opt/redis4/bin/redis-trib.rb:1423:in `create_cluster_cmd' from /opt/redis4/bin/redis-trib.rb:1830:in `send' from /opt/redis4/bin/redis-trib.rb:1830

创建集群报错

 

# /opt/data/redis-4.0.10/src/redis-trib.rb create --replicas 2 10.0.0.71:7000 10.0.0.71:7001 10.0.0.71:7002 10.0.0.72:7003 10.0.0.72:7004 10.0.0.72:7005 10.0.0.73:7006 10.0.0.73:7007 10.0.0.73:7008>>> Creating cluster>>> Performing hash slots allocation on 9 nodes...Using 3 masters:10.0.0.71:700010.0.0.72:700310.0.0.73:7006Adding replica 10.0.0.72:7004 to 10.0.0.71:7000Adding replica 10.0.0.73:7007 to 10.0.0.71:7000Adding replica 10.0.0.71:7002 to 10.0.0.72:7003Adding replica 10.0.0.73:7008 to 10.0.0.72:7003Adding replica 10.0.0.72:7005 to 10.0.0.73:7006Adding replica 10.0.0.71:7001 to 10.0.0.73:7006M: 02729b1998d534a86ce25a5d5dcdb18fdc711384 10.0.0.71:7000 slots:0-5460 (5461 slots) masterS: b84cc1fc82e161222d2b898f1c60c0c8b019e4d6 10.0.0.71:7001 replicates f913a1b6c7513c341ff9dfa25d13a0fd5b776e11S: ffc07311a92f533ddeb1f8db77f0f2c49cce744b 10.0.0.71:7002 replicates 2061b284c2eedd20e1f364348ffa9ea5f8c69704M: 2061b284c2eedd20e1f364348ffa9ea5f8c69704 10.0.0.72:7003 slots:5461-10922 (5462 slots) masterS: efee2ff0fb42e7d492ba8101d5e0ae3a693233ee 10.0.0.72:7004 replicates 02729b1998d534a86ce25a5d5dcdb18fdc711384S: 79c7217d89004999a512f86cfad11c239a400cfe 10.0.0.72:7005 replicates f913a1b6c7513c341ff9dfa25d13a0fd5b776e11M: f913a1b6c7513c341ff9dfa25d13a0fd5b776e11 10.0.0.73:7006 slots:10923-16383 (5461 slots) masterS: 4025f0cd1be2f5e94f3086faebd6528fcbef74bc 10.0.0.73:7007 replicates 02729b1998d534a86ce25a5d5dcdb18fdc711384S: d71f52820c914648a146d039fd36fcf1c217403d 10.0.0.73:7008 replicates 2061b284c2eedd20e1f364348ffa9ea5f8c69704Can I set the above configuration? (type 'yes' to accept): yes>>> Nodes configuration updated>>> Assign a different config epoch to each node>>> Sending CLUSTER MEET messages to join the clusterWaiting for the cluster to join......>>> Performing Cluster Check (using node 10.0.0.71:7000)M: 02729b1998d534a86ce25a5d5dcdb18fdc711384 10.0.0.71:7000 slots:0-5460 (5461 slots) master 2 additional replica(s)S: d71f52820c914648a146d039fd36fcf1c217403d 10.0.0.73:7008 slots: (0 slots) slave replicates 2061b284c2eedd20e1f364348ffa9ea5f8c69704S: 79c7217d89004999a512f86cfad11c239a400cfe 10.0.0.72:7005 slots: (0 slots) slave replicates f913a1b6c7513c341ff9dfa25d13a0fd5b776e11M: f913a1b6c7513c341ff9dfa25d13a0fd5b776e11 10.0.0.73:7006 slots:10923-16383 (5461 slots) master 2 additional replica(s)M: 2061b284c2eedd20e1f364348ffa9ea5f8c69704 10.0.0.72:7003 slots:5461-10922 (5462 slots) master 2 additional replica(s)S: efee2ff0fb42e7d492ba8101d5e0ae3a693233ee 10.0.0.72:7004 slots: (0 slots) slave replicates 02729b1998d534a86ce25a5d5dcdb18fdc711384S: ffc07311a92f533ddeb1f8db77f0f2c49cce744b 10.0.0.71:7002 slots: (0 slots) slave replicates 2061b284c2eedd20e1f364348ffa9ea5f8c69704S: 4025f0cd1be2f5e94f3086faebd6528fcbef74bc 10.0.0.73:7007 slots: (0 slots) slave replicates 02729b1998d534a86ce25a5d5dcdb18fdc711384S: b84cc1fc82e161222d2b898f1c60c0c8b019e4d6 10.0.0.71:7001 slots: (0 slots) slave replicates f913a1b6c7513c341ff9dfa25d13a0fd5b776e11[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384

可以看到有3个主节点，6个从节点。每个节点都是成功的连接状态。

　　　　1、主节点M: 02729b1998d534a86ce25a5d5dcdb18fdc711384 10.0.0.71:7000 slots:0-5460 (5461 slots) master 2 additional replica(s)M: f913a1b6c7513c341ff9dfa25d13a0fd5b776e11 10.0.0.73:7006 slots:10923-16383 (5461 slots) master 2 additional replica(s)M: 2061b284c2eedd20e1f364348ffa9ea5f8c69704 10.0.0.72:7003 slots:5461-10922 (5462 slots) master 2　　　　2、从节点S: d71f52820c914648a146d039fd36fcf1c217403d 10.0.0.73:7008 slots: (0 slots) slave replicates 2061b284c2eedd20e1f364348ffa9ea5f8c69704S: 79c7217d89004999a512f86cfad11c239a400cfe 10.0.0.72:7005 slots: (0 slots) slave replicates f913a1b6c7513c341ff9dfa25d13a0fd5b776e11S: efee2ff0fb42e7d492ba8101d5e0ae3a693233ee 10.0.0.72:7004 slots: (0 slots) slave replicates 02729b1998d534a86ce25a5d5dcdb18fdc711384S: ffc07311a92f533ddeb1f8db77f0f2c49cce744b 10.0.0.71:7002 slots: (0 slots) slave replicates 2061b284c2eedd20e1f364348ffa9ea5f8c69704S: 4025f0cd1be2f5e94f3086faebd6528fcbef74bc 10.0.0.73:7007 slots: (0 slots) slave replicates 02729b1998d534a86ce25a5d5dcdb18fdc711384S: b84cc1fc82e161222d2b898f1c60c0c8b019e4d6 10.0.0.71:7001 slots: (0　　4、关闭集群

　　pkill redis

　　或者

for((i=0;i set my_name linux-> Redirected to slot [12803] located at 10.0.0.73:7006OK10.0.0.73:7006> get my_name"linux"

redis cluster 采用的方式很直接，它直接跳转到7006节点了，而不是还在自身的7000节点

　　　　2、连接7005这个节点获取数据# redis-cli -h Redis02 -c -p 7005Redis02:7005> get my_name-> Redirected to slot [12803] located at 10.0.0.73:7006"linux"

同样是获取my_name的值，它同样也是跳转到了7006上

　　　　3、设置多个值测试10.0.0.72:7003> zadd name 333 rrr(integer) 110.0.0.72:7003> zadd erew 333 rrr(integer) 110.0.0.72:7003> zrange erew 0 -11) "rrr"

　　可以看出，数据都会在7000、7003、7006这3个主节点来跳转存储。 

　　2、测试集群中的slave节点挂掉

　　三个主节点7000、7003、7006提供数据存储和读取，6个从节点7001、7002、7004、7005、7007、7008负责把主节点的数据同步到自己的节点上。

　　查看三个主节点的appendonly.aof的内容，数据是否一致。

# cat appendonly.aof *2$6SELECT$10*3$3set$3age$3123测试挂掉 集群中Master

　　安装前面的理论，7003主节点挂掉了，那么这个时候，7003的从节点只有7003一个，肯定7003就会被选举成 Master 节点了：

# ps -ef|grep [r]edisroot 6640 1 0 16:32 ? 00:00:32 /opt/data/redis-4.0.10/src/redis-server 10.0.0.72:7003 [cluster]root 6642 1 0 16:32 ? 00:00:32 /opt/data/redis-4.0.10/src/redis-server 10.0.0.72:7004 [cluster]root 6650 1 0 16:32 ? 00:00:32 /opt/data/redis-4.0.10/src/redis-server 10.0.0.72:7005 [cluster]# kill 6640[root@redis02 7003]# ps -ef|grep [r]edisroot 6642 1 0 16:32 ? 00:00:32 /opt/data/redis-4.0.10/src/redis-server 10.0.0.72:7004 [cluster]root 6650 1 0 16:32 ? 00:00:33 /opt/data/redis-4.0.10/src/redis-server 10.0.0.72:7005 [cluster]

　　查看选举的结果，一个从节点变成主节点，但这个主节点只有一个从节点了

[root@redis02 7003]# redis-trib.rb check 10.0.0.72:7003[ERR] Sorry, can't connect to node 10.0.0.72:7003[root@redis02 7003]# redis-trib.rb check 10.0.0.72:7004>>> Performing Cluster Check (using node 10.0.0.72:7004)S: efee2ff0fb42e7d492ba8101d5e0ae3a693233ee 10.0.0.72:7004 slots: (0 slots) slave replicates 02729b1998d534a86ce25a5d5dcdb18fdc711384S: ffc07311a92f533ddeb1f8db77f0f2c49cce744b 10.0.0.71:7002 slots: (0 slots) slave replicates d71f52820c914648a146d039fd36fcf1c217403dM: f913a1b6c7513c341ff9dfa25d13a0fd5b776e11 10.0.0.73:7006 slots:10923-16383 (5461 slots) master 2 additional replica(s)S: 4025f0cd1be2f5e94f3086faebd6528fcbef74bc 10.0.0.73:7007 slots: (0 slots) slave replicates 02729b1998d534a86ce25a5d5dcdb18fdc711384S: 79c7217d89004999a512f86cfad11c239a400cfe 10.0.0.72:7005 slots: (0 slots) slave replicates f913a1b6c7513c341ff9dfa25d13a0fd5b776e11S: b84cc1fc82e161222d2b898f1c60c0c8b019e4d6 10.0.0.71:7001 slots: (0 slots) slave replicates f913a1b6c7513c341ff9dfa25d13a0fd5b776e11M: d71f52820c914648a146d039fd36fcf1c217403d 10.0.0.73:7008 slots:5461-10922 (5462 slots) master 1 additional replica(s)M: 02729b1998d534a86ce25a5d5dcdb18fdc711384 10.0.0.71:7000 slots:0-5460 (5461 slots) master 2 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384三、管理脚本　定时进行aof重写bgrewriteaof.sh #!/bin/bash############################################## Functions: start & stop redis cluster node# ChangeLog:# 2018-05-30 [email protected] initial############################################# set -x #DEBUG

REDIS_BIN_DIR="/data/redis/bin"REDIS_CLUSTER_DIR="/data/redis-cluster"REDIS_CLI="$REDIS_BIN_DIR/redis-cli"REDIS_SERVER="$REDIS_BIN_DIR/redis-server"

PORTLIST=(7000 7003)

bgrewriteaof() {

for REDIS_PORT in ${PORTLIST[@]}; do REDIS_NAME="redis-$REDIS_PORT" REDIS_CONFIG="/data/redis-cluster/conf/$REDIS_PORT.conf" REDIS_PASS="Aniuredis123" echo -n $"Rewriteaof $REDIS_NAME: " $REDIS_CLI -p $REDIS_PORT -a $REDIS_PASS BGREWRITEAOF retval=$? [ $retval -eq 0 ] && echo " $REDIS_NAME bgrewriteaof succeed!" done}

　　

redis-7000.sh # redis实例启动脚本#!/bin/bash############################################## Functions: start & stop redis cluster node# ChangeLog:# 2018-05-30 [email protected] initial############################################# set -x #DEBUG

REDIS_BIN_DIR="/data/redis/bin"REDIS_CLUSTER_DIR="/data/redis-cluster"REDIS_CLI="$REDIS_BIN_DIR/redis-cli"REDIS_SERVER="$REDIS_BIN_DIR/redis-server"

REDIS_USER="root"REDIS_PORT=7000REDIS_NAME="redis-$REDIS_PORT"REDIS_CONFIG="/data/redis-cluster/conf/$REDIS_PORT.conf"REDIS_PASS="Aniuredis123"

start() { [ -f $REDIS_CONFIG ] || exit 6 [ -x $REDIS_SERVER ] || exit 5 echo -n $"Starting $REDIS_NAME: " $REDIS_SERVER $REDIS_CONFIG retval=$? [ $retval -eq 0 ] && echo " $REDIS_NAME start succeed!"}

stop() { echo -n $"Stopping $REDIS_NAME: " $REDIS_CLI -p $REDIS_PORT -a $REDIS_PASS shutdown retval=$? [ $retval -eq 0 ] && echo " $REDIS_NAME stop succeed!" }

restart() { stop start}

case "$1" in start) $1 ;; stop) $1 ;; restart) $1 ;; *) echo $"Usage: $0 {start|stop|restart}" exit 2esacexit $?

 



【本文地址】

公司简介

联系我们

今日新闻

    推荐新闻

    专题文章
      CopyRight 2018-2019 实验室设备网 版权所有