RHEL 5.4下部署LVS(DR)+keepalived實現高性能高可用負載均衡
一、簡介
LVS是Linux Virtual Server的簡寫,意即Linux虛擬服務器,是一個虛擬的服務器集群系統。本項目在1998年5月由章文嵩博士成立,是中國國內最早出現的自由軟件項目之一。
目前有三種IP負載均衡技術(VS/NAT、VS/TUN和VS/DR);十種調度算法(rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq)。
Keepalived在這里主要用作RealServer的健康狀態檢查以及LoadBalance主機和BackUP主機之間failover的實現。
LVS+Keepalived能實現的功能:利用LVS控制器主備模式避免單點故障以及自動刪除故障WEB服務器結點并當它恢復后再自動添加到群集中。
拓撲圖:
二、系統環境
系統平臺:RHEL?5.4
LVS版本:ipvsadm-1.24
keepalived版本:keepalived-1.1.15
三、安裝
1、在兩臺LVS Server上安裝lvs、keepalived軟件(LVS_MASTER & LVS_BACKUP)
lvs install -------------
[root@LVS-MASTER ~]# wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz
[root@LVS-MASTER ~]# ln -s /usr/src/kernels/2.6.18-194.el5-i686//usr/src/linux/
[root@LVS-MASTER ~]# tar zxvf ipvsadm-1.24.tar.gz
[root@LVS-MASTER ~]# cd ipvsadm-1.24
[root@LVS-MASTER ipvsadm-1.24]# make && make install
Keepalived install?
-------------
[root@LVS-MASTER ~]# wget http://www.keepalived.org/software/keepalived-1.1.15.tar.gz
[root@LVS-MASTER ~]# tar zxvf keepalived-1.1.15.tar.gz
[root@LVS-MASTER ~]# cd keepalived-1.1.15
[root@LVS-MASTER keepalived-1.1.15]# ./configure && make && make install
#########?將keepalived做成啟動服務,方便管理##########
[root@LVS-MASTER ~]# cp /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/
[root@LVS-MASTER ~]# cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
[root@LVS-MASTER ~]# mkdir /etc/keepalived/
[root@LVS-MASTER ~]# cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
[root@LVS-MASTER ~]# cp /usr/local/sbin/keepalived /usr/sbin/
[root@LVS-MASTER ~]# service keepalived start | stop
2、keepalived的配置
###################? MASTER ###################
! Configuration File for keepalived
global_defs {
notification_email {
root@linux.tang.chao #設置報警郵件地址,可以設置多個,每行一個。
mchina_tang@qq.com #需開啟本機的sendmail服務
}
notification_email_from keepalived@localhost #設置郵件的發送地址
smtp_server 127.0.0.1 #設置smtp server地址
smtp_connect_timeout 30 #設置連接smtp server的超時時間
router_id LVS_DEVEL #表示運行keepalived服務器的一個標識。發郵件時顯示在郵件主題的信息
}
vrrp_instance VI_1 {
state MASTER #指定keepalived的角色,MASTER表示此主機是主服務器,BACKUP表示此主機是備用服務器
interface eth0 #指定HA監測網絡的接口
virtual_router_id 51 #虛擬路由標識,這個標識是一個數字,同一個vrrp實例使用唯一的標識。即同一vrrp_instance下,MASTER和BACKUP必須是一致的
priority 100 #定義優先級,數字越大,優先級越高,在同一個vrrp_instance下,MASTER的優先級必須大于BACKUP的優先級
advert_int 1 #設定MASTER與BACKUP負載均衡器之間同步檢查的時間間隔,單位是秒
authentication { #設置驗證類型和密碼
auth_type PASS #設置驗證類型,主要有PASS和AH兩種
auth_pass 1111 #設置驗證密碼,在同一個vrrp_instance下,MASTER與BACKUP必須使用相同的密碼才能正常通信
}
virtual_ipaddress { #設置虛擬IP地址,可以設置多個虛擬IP地址,每行一個
10.0.0.148
}
}
virtual_server 10.0.0.148 80 { #設置虛擬服務器,需要指定虛擬IP地址和服務端口,IP與端口之間用空格隔開
delay_loop 6 #設置運行情況檢查時間,單位是秒
lb_algo rr #設置負載調度算法,這里設置為rr,即輪詢算法
lb_kind DR #設置LVS實現負載均衡的機制,有NAT、TUN、DR三個模式可選
persistence_timeout 50 #會話保持時間,單位是秒。這個選項對動態網頁是非常有用的,為集群系統中的session共享提供了一個很好的解決方案。
#有了這個會話保持功能,用戶的請求會被一直分發到某個服務節點,直到超過這個會話的保持時間。
#需要注意的是,這個會話保持時間是最大無響應超時時間,也就是說,用戶在操作動態頁面時,如果50秒內沒有執行任何操作,
#那么接下來的操作會被分發到另外的節點,但是如果用戶一直在操作動態頁面,則不受50秒的時間限制
protocol TCP #指定轉發協議類型,有TCP和UDP兩種
real_server 10.0.0.137 80 { #配置服務節點1,需要指定real server的真實IP地址和端口,IP與端口之間用空格隔開
weight 3 #配置服務節點的權值,權值大小用數字表示,數字越大,權值越高,設置權值大小可以為不同性能的服務器
#分配不同的負載,可以為性能高的服務器設置較高的權值,而為性能較低的服務器設置相對較低的權值,這樣才能合理地利用和分配系統資源
TCP_CHECK { #realserver的狀態檢測設置部分,單位是秒
connect_timeout 10 #表示3秒無響應超時
nb_get_retry 3 #表示重試次數
delay_before_retry 3 #表示重試間隔
connect_port 80
}
}
real_server 10.0.0.139 80 {
weight 3
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
####################?BACKUP ###################
! Configuration File for keepalived
global_defs {
notification_email {
root@linux.tang.chao
mchina_tang@qq.com
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.148
}
}
virtual_server 10.0.0.148 80 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 50
protocol TCP
real_server 10.0.0.137 80 {
weight 3
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 10.0.0.139 80 {
weight 3
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
3、在兩臺Web Server上執行realserver.sh腳本,為lo:0綁定VIP地址10.0.0.148、抑制ARP廣播。
[root@web1 ~]# cat realserver.sh
#!/bin/bash #description: Config realserver VIP=10.0.0.148 /etc/rc.d/init.d/functions case "$1" in start) /sbin/ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP /sbin/route add -host $VIP dev lo:0 echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce sysctl -p >/dev/null 2> &1 echo "RealServer Start OK" ;; stop) /sbin/ifconfig lo:0 down /sbin/route del $VIP >/dev/null 2> &1 echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce echo "RealServer Stoped" ;; *) echo "Usage: $0 {start|stop}" exit 1 esac exit 0
[root@web1 ~]# sh realserver.sh start
4、分別在LVS-MASTER、LVS-BACKUP上執行service keepalived start啟動keepalived就可實現負載均衡及高可用集群了;
[root@LVS-MASTER keepalived]# service keepalived start
[root@LVS-MASTER keepalived]# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
? -> RemoteAddress:Port?????????? Forward Weight ActiveConn InActConn
TCP? 10.0.0.148:http rr persistent 50
? -> 10.0.0.139:http????????????? Route?? 3????? 0????????? 0??????? ?
? -> 10.0.0.137:http????????????? Route?? 3????? 0????????? 0??????? ?
[root@LVS-MASTER keepalived]#
查看日志/var/log/messages
[root@LVS-MASTER keepalived]# tail -n 30/var/log/messages
Apr 19 03:59:58 LVS-MASTER avahi-daemon[2845]: Withdrawing address record for10.0.0.148 on eth0.
Apr 19 04:00:19 LVS-MASTER Keepalived: Starting Keepalived v1.2.1 (04/17,2012)
Apr 19 04:00:19 LVS-MASTER Keepalived: Starting Healthcheck child process,pid=5960
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Netlink reflector reportsIP 10.0.0.136 added
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Netlink reflector reportsIP 10.0.0.159 added
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Registering Kernelnetlink reflector
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Registering Kernelnetlink command channel
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Opening file '/etc/keepalived/keepalived.conf'.
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Configuration is using :12257 Bytes
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Using LinkWatch kernelnetlink reflector...
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Activating healtcheckerfor service [10.0.0.137:80]
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Activating healtcheckerfor service [10.0.0.139:80]
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Netlink reflector reports IP10.0.0.136 added
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Netlink reflector reports IP10.0.0.159 added
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Registering Kernel netlinkreflector
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Registering Kernel netlink commandchannel
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Registering gratutious ARP sharedchannel
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Opening file'/etc/keepalived/keepalived.conf'.
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Configuration is using : 36698Bytes
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Using LinkWatch kernel netlinkreflector...
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: VRRP sockpool: [ifindex(2),proto(112), fd(11,12)]
Apr 19 04:00:19 LVS-MASTER Keepalived: Starting VRRP child process, pid=5961
Apr 19 04:00:20 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) Transition toMASTER STATE
Apr 19 04:00:21 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTERSTATE
Apr 19 04:00:21 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) settingprotocol VIPs.
Apr 19 04:00:21 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 10.0.0.148
Apr 19 04:00:21 LVS-MASTER Keepalived_healthcheckers: Netlink reflector reportsIP 10.0.0.136 added
Apr 19 04:00:21 LVS-MASTER avahi-daemon[2845]: Registering new address recordfor 10.0.0.148 on eth0.
Apr 19 04:00:21 LVS-MASTER Keepalived_vrrp: Netlink reflector reports IP10.0.0.136 added
Apr 19 04:00:26 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 10.0.0.148
[root@LVS-MASTER keepalived]#
四、測試
接下來做高可用性測試&故障切換測試......
####高可用性測試####
模擬故障,將LVS-MASTER上的keepalived服務停掉,然后觀察LVS-BACKUP上的日志,信息如下
[root@LVS-BACKUP keepalived]# tail -f /var/log/messages
Apr 19 03:33:04 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1)Transition to MASTER STATE
Apr 19 03:33:05 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTERSTATE
Apr 19 03:33:05 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) settingprotocol VIPs.
Apr 19 03:33:05 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 10.0.0.148
Apr 19 03:33:05 LVS-BACKUP Keepalived_vrrp: Netlink reflector reports IP10.0.0.157 added
Apr 19 03:33:05 LVS-BACKUP Keepalived_healthcheckers: Netlink reflector reportsIP 10.0.0.157 added
Apr 19 03:33:05 LVS-BACKUP avahi-daemon[2825]: Registering new address recordfor 10.0.0.148 on eth0.
Apr 19 03:33:10 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 10.0.0.148
從日志中可知,主機出現故障后,備機立刻檢測到,此時備機變為MASTER角色,并且接管了主機的虛擬IP資源,最后將虛擬IP綁定在etho設備上。
將LVS-MASTER?上的keepalived服務開啟后,LVS-BACKUP的日志狀態。
Apr 19 03:34:02 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) Received higher prio advert
Apr 19 03:34:02 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1)Entering BACKUP STATE
Apr 19 03:34:02 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.
Apr 19 03:34:02 LVS-BACKUP Keepalived_vrrp: Netlink reflector reports IP 10.0.0.157 removed
Apr 19 03:34:02 LVS-BACKUP Keepalived_healthcheckers: Netlink reflector reports IP 10.0.0.157 removed
Apr 19 03:34:02 LVS-BACKUP avahi-daemon[2825]: Withdrawing address record for 10.0.0.148 on eth0.
從日志可知,備機在檢測到主機重新恢復正常后,釋放了虛擬IP資源重新成為BACKUP角色
####故障切換測試####
故障切換是測試當某個節點出現故障后,Keepalived監制模塊是否能及時發現然后屏蔽故障節點,同時將服務器轉移到正常節點來執行。
將web2節點服務停掉,假設這個節點出現故障,然后主、備機日志信息如下
Apr 19 03:35:04 LVS-MASTER Keepalived_healthcheckers:TCP connection to [10.0.0.139:80] failed!!!
Apr 19 03:35:04 LVS-MASTER Keepalived_healthcheckers:Removing service [10.0.0.139:80] from VS [10.0.0.148:80]
Apr 19 03:35:04 LVS-MASTER Keepalived_healthcheckers: Remote SMTP server [127.0.0.1:25] connected.
[root@LVS-MASTER keepalived]# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
? -> RemoteAddress:Port?????????? Forward Weight ActiveConn InActConn
TCP? 10.0.0.148:http rr persistent 50
? -> 10.0.0.137:http????????????? Route?? 3????? 0????????? 0??????? ?
[root@LVS-MASTER keepalived]#
從以上可以看出,Keepalived監控模塊檢測到10.0.0.139這臺主機出現故障后,將些web2從集群系統中剔除掉了。?此時訪問http://10.0.0.148只能看到web1了)
重新啟動web2節點的服務,日志信息如下:
Apr 19 03:38:22 LVS-MASTER Keepalived_healthcheckers:TCP connection to [10.0.0.139:80] success.
Apr 19 03:38:22 LVS-MASTER Keepalived_healthcheckers:Adding service [10.0.0.139:80] to VS [10.0.0.148:80]
Apr 19 03:38:22 LVS-MASTER Keepalived_healthcheckers: Remote SMTP server [127.0.0.1:25] connected.
[root@LVS-MASTERkeepalived]# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
? -> RemoteAddress:Port?????????? Forward Weight ActiveConn InActConn
TCP? 10.0.0.148:http rr persistent 50
? -> 10.0.0.139:http????????????? Route?? 3????? 0????????? 0??????? ?
? -> 10.0.0.137:http????????????? Route?? 3????? 0????????? 0??????? ?
[root@LVS-MASTER keepalived]#
Keepalived監控模塊檢測到10.0.0.139這臺主機恢復正常后,又將此節點加入集群系統中,再次訪問就可以訪問到web2頁面了)
?
CentOS 6.3下部署LVS(NAT)+keepalived實現高性能高可用負載均衡
http://www.cnblogs.com/mchina/archive/2012/08/27/2644391.html
更多文章、技術交流、商務合作、聯系博主
微信掃碼或搜索:z360901061

微信掃一掃加我為好友
QQ號聯系: 360901061
您的支持是博主寫作最大的動力,如果您喜歡我的文章,感覺我的文章對您有幫助,請用微信掃描下面二維碼支持博主2元、5元、10元、20元等您想捐的金額吧,狠狠點擊下面給點支持吧,站長非常感激您!手機微信長按不能支付解決辦法:請將微信支付二維碼保存到相冊,切換到微信,然后點擊微信右上角掃一掃功能,選擇支付二維碼完成支付。
【本文對您有幫助就好】元
