nginx+keepalived

一、简介

很久没有配置keepalived了，最近业务医院需要配置前置机做转发，并且配置keepalived.
因为医院的生产环境和外界是完全隔离的，所以需要做前置机做网络请求转发

二、服务器列表

角色	IP	备注
nginx + keepalived master	192.168.1.22
nginx + keepalived slave	192.168.1.32
vip	192.168.1.100

三、安装nginx

1.安装依赖

yum -y install gcc gcc-c++ zlib zlib-devel pcre pcre-devel zlib zlib-devel openssl openssl-devel

2.安装

./configure --prefix=/usr/local/nginx --with-http_ssl_module --with-http_realip_module  --with-http_gzip_static_module --with-pcre --with-http_stub_status_module --with-http_dav_module --with-http_addition_module --with-http_sub_module --with-http_flv_module --with-http_mp4_module --with-streammake && make installn -s /usr/local/nginx/sbin/nginx /usr/local/sbin/

3.配置启动脚本

vim /usr/lib/systemd/system/nginx.service 
[Unit]
Description=nginx
After=network.target[Service]
Type=forking
ExecStart=/usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf
ExecReload=/usr/local/nginx/sbin/nginx -s reload
ExecStop=/usr/local/nginx/sbin/nginx -s quit
PrivateTmp=true[Install]
WantedBy=multi-user.target

四、安装keepalived

yum -y install keepalived

五、配置master

配置1.22的keepalived

1.配置keepalived

vim /etc/keepalived/keepalived.conf! Configuration File for keepalived
global_defs {# 路由id：当前安装keepalived的节点主机标识符，保证全局唯一router_id Internet_MASTERscript_user rootenable_script_security
}
vrrp_script check_nginx {script "/etc/keepalived/check_nginx.sh"interval 2 	# 每隔两秒运行上一行脚本weight -5 	# 如果脚本运行成功，则升级权重+5 ; weight -5 # 如果脚本运行失败，则升级权重-5fall 2  	#检测连续两次失败才算真的失败rise 1  	#检测1次成功就算成功
}
vrrp_instance VI_10 {state MASTER 		# 表示状态是MASTER主机还是备用机BACKUPinterface ens33 		# 该实例绑定的网卡名称virtual_router_id 100 	# 保证主备节点一致即可,这个其实就是虚拟出来路由器的id，id一样才能说明所有节点在一个组priority 51 		# 权重，master权重一般高于backup，如果有多个，那就是选举，谁的权重高，谁就当选advert_int 1 		# 主备之间同步检查时间间隔，单位秒authentication {  		# 认证权限密码，防止非法节点进入auth_type PASSauth_pass 2222}virtual_ipaddress {  	# 虚拟出来的ip，可以有多个（vip）192.168.1.100/24 dev ens33 label ens33:1}track_script {check_nginx   # 追踪nginx脚本}
}

2.配置检测脚本

vim /etc/keepalived/check_nginx.sh#!/bin/bash
nginx_stat=$(ps axu |grep -w nginx|wc -l)
# 判断nginx是否宕机，如果宕机了，尝试重启
if [ $nginx_stat -eq 1 ];thensystemctl start nginx# 等待一小会再次检查nginx，如果没有启动成功，则停止keepalived，使其启动备用机sleep 3if [ $(ps axu |grep -w nginx |wc -l) -eq 1 ];thensystemctl stop keepalivedfi
f

六、配置backup

配置1.32服务器

1.配置keepalived

! Configuration File for keepalived
global_defs {# 路由id：当前安装keepalived的节点主机标识符，保证全局唯一router_id Internet_BACKUPscript_user rootenable_script_security
}
vrrp_script check_nginx {script "/etc/keepalived/check_nginx.sh"interval 2 	# 每隔两秒运行上一行脚本weight -5 	# 如果脚本运行成功，则升级权重+5 ; weight -5 # 如果脚本运行失败，则升级权重-5fall 2  	#检测连续两次失败才算真的失败rise 1  	#检测1次成功就算成功
}
vrrp_instance VI_10 {state BACKUP 		# 表示状态是MASTER主机还是备用机BACKUPinterface ens33 		# 该实例绑定的网卡名称virtual_router_id 100 	# 保证主备节点一致即可,这个其实就是虚拟出来路由器的id，id一样才能说明所有节点在一个组priority 50 		# 权重，master权重一般高于backup，如果有多个，那就是选举，谁的权重高，谁就当选advert_int 1 		# 主备之间同步检查时间间隔，单位秒authentication {  		# 认证权限密码，防止非法节点进入auth_type PASSauth_pass 2222}virtual_ipaddress {  	# 虚拟出来的ip，可以有多个（vip）192.168.1.100/24 dev ens33 label ens33:1}track_script {check_nginx   # 追踪nginx脚本}
}

2.配置检测脚本

#!/bin/bash
nginx_stat=$(ps axu |grep -w nginx|wc -l)
if [ $nginx_stat -le 1 ];thennginxsleep 3if [ $(ps axu |grep -w nginx |wc -l) -le 1 ];thensystemctl stop keepalivedfi
fi

七、启动keepalived

1.测试检测脚本

两台机器分别进行启动测试
nginx启动成功，说明脚本生效

[root@node4 keepalived]# sh check_nginx.sh 
[root@node4 keepalived]# ps aux |grep nginx
root      19245  0.0  0.0  20708   636 ?        Ss   04:36   0:00 nginx: master process nginx
nobody    19247  0.0  0.1  21148  1080 ?        S    04:36   0:00 nginx: worker process
root      19253  0.0  0.0 112812   980 pts/0    R+   04:36   0:00 grep --color=auto nginx

2.测试keepalived

分别启动主备服务器
systemctl start keepalived

在1.22服务器上查看

[root@node4 keepalived]# ifconfig 
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500inet 192.168.1.22  netmask 255.255.255.0  broadcast 192.168.1.255inet6 fe80::aafa:3af8:b0c0:5970  prefixlen 64  scopeid 0x20<link>ether 00:0c:29:fb:fe:b6  txqueuelen 1000  (Ethernet)RX packets 291331  bytes 376368562 (358.9 MiB)RX errors 0  dropped 0  overruns 0  frame 0TX packets 138247  bytes 118532393 (113.0 MiB)TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0ens33:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500inet 192.168.1.100  netmask 255.255.255.0  broadcast 0.0.0.0ether 00:0c:29:fb:fe:b6  txqueuelen 1000  (Ethernet)

在1.32服务器上查看，没有1.100的vip

[root@localhost ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000link/ether 00:0c:29:8a:ba:00 brd ff:ff:ff:ff:ff:ffinet 192.168.1.32/24 brd 192.168.1.255 scope global noprefixroute dynamic ens32valid_lft 1614sec preferred_lft 1614secinet6 fe80::cf4e:8df6:7cb2:e053/64 scope link noprefixroute valid_lft forever preferred_lft forever

3.测试check_nginx脚本实时性

keepalived是2秒钟运行一次脚本，在配置文件有配置，这里进行测试一下。
先看查nginx状态

[root@node4 ~]# ps aux |grep nginx
root      19245  0.0  0.0  20708   636 ?        Ss   04:36   0:00 nginx: master process nginx
nobody    19247  0.0  0.1  21148  1080 ?        S    04:36   0:00 nginx: worker process
root      51295  0.0  0.0 112812   976 pts/0    R+   20:26   0:00 grep --color=auto nginx

停止nginx后，查看keepalived会不会通过脚本来启动nginx.
这里发现nginx的pid发生了变化，说明nginx是keepalived通过脚本进行启动了

[root@node4 ~]# nginx -s stop
[root@node4 ~]# ps aux |grep nginx
root      51382  0.0  0.0  20708   628 ?        Ss   20:26   0:00 nginx: master process nginx
nobody    51384  0.0  0.1  21144  1068 ?        S    20:26   0:00 nginx: worker process
root      51506  0.0  0.0 112812   980 pts/0    R+   20:27   0:00 grep --color=auto nginx

4.测试漂移功能

停止master（1.22）的keepalived

[root@node4 ~]# systemctl  stop keepalived
[root@node4 ~]# ps aux |grep keep
root      51815  0.0  0.0 112812   976 pts/0    R+   20:29   0:00 grep --color=auto keep

此时发现vip已经漂移到了1.32服务器上

2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000link/ether 00:0c:29:8a:ba:00 brd ff:ff:ff:ff:ff:ffinet 192.168.1.32/24 brd 192.168.1.255 scope global noprefixroute dynamic ens32valid_lft 1788sec preferred_lft 1788secinet 192.168.1.100/24 scope global secondary ens32:1valid_lft forever preferred_lft foreverinet6 fe80::cf4e:8df6:7cb2:e053/64 scope link noprefixroute valid_lft forever preferred_lft forever

然后恢复master的keepalived，发现vip已经漂移到了1.22服务器上。1.32服务器上没有了vip

[root@node4 ~]# ip a |grep ens33
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000inet 192.168.1.22/24 brd 192.168.1.255 scope global dynamic ens33inet 192.168.1.100/24 scope global secondary ens33:1