配置文件准备
3主3从 一共6个redis 服务
每个文件夹放入一个 redis.conf 文件
端口按文件夹替换
port 7291
daemonize yes
protected-mode no
dir /usr/local/soft/redis-5.0.5/redis-cluster/7291/
cluster-enabled yes
cluster-config-file nodes-7291.conf
cluster-node-timeout 5000
appendonly yes
pidfile /var/run/redis_7291.pid
如果是外网要加如下配置:
cluster-announce-ip 479.xx.xx.xx
# 节点映射端口
cluster-announce-port ${PORT}
# 节点总线端口
cluster-announce-bus-port 1${PORT}
启动脚本
- redis-server 7291/redis.conf
- redis-server 7292/redis.conf
- redis-server 7293/redis.conf
- redis-server 7294/redis.conf
- redis-server 7295/redis.conf
- redis-server 7296/redis.conf
检查启动完成的进程
ps -ef|grep redis
创建集群:
ip 不要使用 127.0.0.1
redis-cli --cluster create 192.168.1.149:7291 192.168.1.149:7292 192.168.1.149:7293 192.168.1.149:7294 192.168.1.149:7295 192.168.1.149:7296 --cluster-replicas 1
redis 会给个预计的方案,6个节点分配3主3从,输入 yes 确认。
槽分布:
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
zhangburui@ZPR-MacBook-Pro cluster % redis-cli --cluster create 192.168.1.149:7291 192.168.1.149:7292 192.168.1.149:7293 192.168.1.149:7294 192.168.1.149:7295 192.168.1.149:7296 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.1.149:7295 to 192.168.1.149:7291
Adding replica 192.168.1.149:7296 to 192.168.1.149:7292
Adding replica 192.168.1.149:7294 to 192.168.1.149:7293
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: c000a3cb2c201baacdb83601deb31a0b299a076e 192.168.1.149:7291
slots:[0-5460] (5461 slots) master
M: 97abcc039fc30ee6ae3a06fed3d1705b4b481299 192.168.1.149:7292
slots:[5461-10922] (5462 slots) master
M: dba3eb1bf1b112ace17697ad4f9ef5f5270e8e42 192.168.1.149:7293
slots:[10923-16383] (5461 slots) master
S: 1117229f2084c42cd964ca84f3224a46cc06b24c 192.168.1.149:7294
replicates 97abcc039fc30ee6ae3a06fed3d1705b4b481299
S: ecf5e306ca20ed285517880e128dcef8c6815245 192.168.1.149:7295
replicates dba3eb1bf1b112ace17697ad4f9ef5f5270e8e42
S: b0d9777198b373a6c52c7738d409b6f305f382db 192.168.1.149:7296
replicates c000a3cb2c201baacdb83601deb31a0b299a076e
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
...
>>> Performing Cluster Check (using node 192.168.1.149:7291)
M: c000a3cb2c201baacdb83601deb31a0b299a076e 192.168.1.149:7291
slots:[0-5460] (5461 slots) master
1 additional replica(s)
S: b0d9777198b373a6c52c7738d409b6f305f382db 192.168.1.149:7296
slots: (0 slots) slave
replicates c000a3cb2c201baacdb83601deb31a0b299a076e
S: 1117229f2084c42cd964ca84f3224a46cc06b24c 192.168.1.149:7294
slots: (0 slots) slave
replicates 97abcc039fc30ee6ae3a06fed3d1705b4b481299
M: 97abcc039fc30ee6ae3a06fed3d1705b4b481299 192.168.1.149:7292
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: ecf5e306ca20ed285517880e128dcef8c6815245 192.168.1.149:7295
slots: (0 slots) slave
replicates dba3eb1bf1b112ace17697ad4f9ef5f5270e8e42
M: dba3eb1bf1b112ace17697ad4f9ef5f5270e8e42 192.168.1.149:7293
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
重置集群的方式是在每个节点上个执行cluster reset,然后重新创建集群
测试脚本:
#!/bin/bash
for ((i=0;i<20000;i++))
do
echo -en "helloworld" | redis-cli -h 192.168.1.149 -p 7291 -c -x set name$i >>redis.log
done
chmod 777 testsetkey.sh
./testsetkey.sh
部分 key 可以插入进去,后续出现下面的问题。
但是 Cluster 是成功了。
查看 各个节点的key 分布情况
127.0.0.1:7292> dbsize
(integer) 6683
127.0.0.1:7293> dbsize
(integer) 6665
127.0.0.1:7291> dbsize
(integer) 6652
出现如下问题?
Could not connect to Redis at 192.168.1.149:7291: Can't assign requested address
Could not connect to Redis at 192.168.1.149:7291: Can't assign requested address
Could not connect to Redis at 192.168.1.149:7291: Can't assign requested address
Could not connect to Redis at 192.168.1.149:7291: Can't assign requested address
Could not connect to Redis at 192.168.1.149:7291: Can't assign requested address
Could not connect to Redis at 192.168.1.149:7291: Can't assign requested address
Could not connect to Redis at 192.168.1.149:7291: Can't assign requested address
Could not connect to Redis at 192.168.1.149:7291: Can't assign requested address
Could not connect to Redis at 192.168.1.149:7291: Can't assign requested address
Could not connect to Redis at 192.168.1.149:7291: Can't assign requested address
Could not connect to Redis at 192.168.1.149:7291: Can't assign requested address
Could not connect to Redis at 192.168.1.149:7291: Can't assign requested address
对redis连续进行50000访问,却出现以了Cannot assignrequestedaddress的问题,我起先是以为是redis的问题(可能承受不了这么多访问量),可是redis被大家吹的那么NB,不会出现这么SB的问题吧,于是google之,发现原来是:
客户端频繁的连服务器,由于每次连接都在很短的时间内结束,导致很多的TIME_WAIT,以至于用光了可用的端口号,所以新的连接没办法绑定端口,即“Cannot assign requestedaddress”。是客户端的问题不是服务器端的问题。通过netstat,的确看到很多TIME_WAIT状态的连接。
从网上找了解决办法:
执行命令修改如下2个内核参数
sysctl -w net.ipv4.tcp_timestamps=1 开启对于TCP时间戳的支持,若该项设置为0,则下面一项设置不起作用
sysctl -w net.ipv4.tcp_tw_recycle=1 表示开启TCP连接中TIME-WAIT sockets的快速回收
类型 | 命令 |
---|---|
集群 | cluster info :打印集群的信息 cluster nodes :列出集群当前已知的所有节点(node),以及这些节点的相关信息。 |
节点 | cluster meet |
槽(slot) | cluster addslots |
键 | cluster keyslot |
评论区