亚洲免费在线-亚洲免费在线播放-亚洲免费在线观看-亚洲免费在线观看视频-亚洲免费在线看-亚洲免费在线视频

集群LVS+GFS+ISCSI+TOMCAT

系統(tǒng) 2581 0
?LVS是中國(guó)人發(fā)起的項(xiàng)目,真是意外呀!大家可以看http://www.douzhe.com/linuxtips/1665.html
我是從最初的HA(高可用性)開始的,別人的例子是用VMWARE,可以做試驗(yàn)但不能實(shí)際應(yīng)用,我又沒有光纖卡的Share Storage,于是就選用ISCSI,成功后又發(fā)現(xiàn)ISCSI+EXT3不能用于LVS,倒最后發(fā)現(xiàn)GFS可用,我最終成功配成可實(shí)際應(yīng)用的LVS,前后斷斷續(xù)續(xù)花了四個(gè)月,走了很多彎路。我花了三天時(shí)間寫下這篇文章,希望對(duì)大家有用。
這里要感謝linuxfans.org、linuxsir.com、chinaunix.com以及其它很多網(wǎng)站,很多資料都是從他們的論壇上找到的。參考文檔及下載點(diǎn)
a.http://www.gyrate.org/misc/gfs.txt
b.http://www.redhat.com/docs/manuals/enterprise/RHEL-3-Manual/cluster-suite/index.html
http://www.redhat.com/docs/manuals/csgfs/admin-guide/index.html
c.ftp://ftp.redhat.com/pub/redhat/linux/updates/enterprise/3ES/en/RHGFS/SRPMS
d.http://distro.ibiblio.org/pub/linux/distributions/caoslinux/centos/3.1/contrib/i386/RPMS/
???????????????? LVS結(jié)構(gòu)圖:??????????????eth0=10.3.1.101????????????eth0:1=10.3.1.254????????????????Load Balance?????????????????? Router??????????????eth1=192.168.1.71????????????eth1:1=192.168.1.1??????????????????|???????? |??????????????????|???????? |????????????????Real1??????Real2?????? eth0=192.168.1.68??eth0=192.168.1.67????????????(eth0 gateway=192.168.1.1)?????? eth1=192.168.0.1---eth1=192.168.0.2??????????????????(雙機(jī)互聯(lián)線)?????????????????????? |?????????????????????? |??????????????????????GFS???????????????????? ISCSI????????????????Share storage??????????????eth0=192.168.1.124
1.Setup ISCSI Server
Server: PIII 1.4,512M, Dell 1650,Redhat 9,IP=192.168.1.124
從http://iscsitarget.sourceforge.net/下載ISCSI TARGET的Source code
(http://sourceforge.net/project/showfiles.php?group_id=108475&package_id=117141)
我選了iscsitarget-0.3.8.tar.gz,要求kernel 2.4.29
從kernel.org下載kernel 2.4.29,解開編譯重啟后編譯安裝iscsitarget-0.3.8:
#make KERNELSRC=/usr/src/linux-2.4.29
#make KERNELSRC=/usr/src/linux-2.4.29 install
#cp ietd.conf /etc
#vi /etc/ietd.conf

# Example iscsi target configuration## Everything until the first target definition belongs# to the global configuration.# Right now this is only the user configuration used# during discovery sessions:# Users, who can access this target# (no users means anyone can access the target)User iscsiuser 1234567890abcTarget iqn.2005-04.com.my:storage.disk2.sys1.iraw1????????User iscsiuser 1234567890abc????????Lun 0 /dev/sda5 fileio????????Alias iraw1Target iqn.2005-04.com.my:storage.disk2.sys1.iraw2????????User iscsiuser 1234567890abc????????Lun 1 /dev/sda6 fileio????????Alias iraw2Target iqn.2005-04.com.my:storage.disk2.sys2.idisk????????User iscsiuser 1234567890abc????????Lun 2 /dev/sda3 fileio????????Alias idiskTarget iqn.2005-04.com.my:storage.disk2.sys2.icca????????User iscsiuser 1234567890abc????????Lun 3 /dev/sda7 fileio????????Alias icca

說明:password 長(zhǎng)度必須不小于12個(gè)字符, Alias是別名, 不知為何這個(gè)別名在Client端顯示不出來.
分區(qū):我只有一個(gè)SCSI盤,所以:

/dev/sda3: Share storage,容量越大越好/dev/sda5: raw1, 建Cluster要的rawdevice, 我給了900M/dev/sda6: raw2, 建Cluster要的rawdevice, 我給了900M/dev/sda7: cca, 建GFS要的,我給了64M(/dev/sda4是Extended分區(qū),在其中建了sda5,6,7)

#Reboot,用service iscsi-target start啟ISCSI server(我覺得比建議的好,可以
用service iscsi-target status看狀態(tài))
2.Setup ISCSI Client(on two real server)
Server: PIII 1.4,512M, Dell 1650,Redhat AS3U4(用AS3U5更好),2.4.21-27.EL
#vi /etc/iscsi.conf

DiscoveryAddress=192.168.1.124OutgoingUsername=iscsiuserOutgoingPassword=1234567890abcUsername=iscsiuserPassword=1234567890abcLoginTimeout=15IncomingUsername=iscsiuserIncomingPassword=1234567890abcSendAsyncTest=yes

#service iscsi restart
#iscsi-ls -l
..., 精簡(jiǎn)如下:
/dev/sdb:iraw2
/dev/sdc:iraw1
/dev/sdd:idisk
/dev/sde:icca
注意: 在real server中ISCSI device的順序很重要,兩個(gè)real server中一定要一樣,如不一樣
   就改ISCSI Server中的設(shè)置,多試幾次
3.Install Redhat Cluster suite
先下載Cluster Suite的ISO, AS3的我是從ChinaUnix.net找到的下載點(diǎn), 安裝clumanager和
redhat-config-cluster。沒有Cluster Suite的ISO也沒關(guān)系,從
ftp://ftp.redhat.com/pub/redhat/linux/updates/enterprise/3ES/en/RHCS/SRPMS/下載
clumanager-1.2.xx.src.rpm,redhat-config-cluster-1.0.x.src.rpm,編譯后安裝,應(yīng)該更好:
#rpm -Uvh clumanager-1.2.26.1-1.src.rpm
#rpmbuild -bs /usr/src/redhat/SPECS/clumanager.spec
#rpmbuild --rebuild --target i686 /usr/src/redhat/SRPMS/clumanager-1.2.26.1-1.src.rpm
還有redhat-config-cluster-1.0.x.src.rpm,也裝好
4.Setup Cluster as HA module
詳細(xì)步驟我就不寫了,網(wǎng)上有很多文章,我也是看了別人的文章學(xué)會(huì)的,不過人家是用VMWARE,
而我是用真的機(jī)子+ISCSI,raw device就是/dev/sdb,/dev/sdc, 然后就
mount /dev/sdd /u01, mkfs.ext3 /u01 ......
設(shè)好后會(huì)發(fā)現(xiàn)ISCSI有問題:同時(shí)只能有一個(gè)Client聯(lián)接寫盤,如果
兩個(gè)Client同時(shí)聯(lián)ISCSI的Share Storge,一個(gè)Client寫,另一個(gè)Client是看不到的,而且此時(shí)文
件系統(tǒng)已經(jīng)破壞了,Client重聯(lián)ISCSI時(shí)會(huì)發(fā)現(xiàn)文件是壞的,用fsck也修復(fù)不了。
ISCSI真的是雞肋嗎?
 NO!從GOOGLE上我終于查到ISCSI只有用Cluster File System才能真正用于Share Storage!
而Redhat買下的GFS就是一個(gè)!
5.Setup GFS on ISCSI
GFS只有Fedora Core4才自帶了,而GFS又一定要用到Cluster Suite產(chǎn)生的/etc/cluster.xml文件,
我沒見FC4有Cluster Suite,真不知Redhat給FC4帶GFS干嘛,饞人嗎?
好,閑話少說,下載:c處的GFS-6.0.2.20-2.src.rpm,按a處的gfs.txt編譯安裝,不過關(guān)于
cluster.ccs,fence.ccs,nodes.ccs的設(shè)置沒說,看b的文檔,我總算弄出來了,都存在
/root/cluster下,存在別的地方也行,不過我不知道有沒有錯(cuò),我沒有光纖卡,文檔又沒講ISCSI
的例子,不過GFS能啟動(dòng)的。
#cat cluster.ccs

cluster {????????name = "Cluster_1"????????lock_gulm {????????????servers = ["cluster1", "cluster2"]????????????heartbeat_rate = 0.9????????????allowed_misses = 10????????}}

注:name就是Cluster Suite設(shè)置的Cluster name, servers就是Cluster member的Hostname,別忘
了加進(jìn)/etc/hosts;allowed_misses我開始設(shè)為1,結(jié)果跑二天GFS就會(huì)死掉,改為10就沒死過了。
#cat fence.ccs

fence_devices{????????admin {??????????????agent = "fence_manual"????????}}

#cat nodes.ccs

nodes {?? cluster1 {??????ip_interfaces {???????? hsi0 = "192.168.0.1"??????}??????fence {???????? human {????????????admin {?????????????? ipaddr = "192.168.0.1"????????????}???????? }??????}?? }?? cluster2 {??????ip_interfaces {???????? hsi0 = "192.168.0.2"??????}??????fence {???????? human {????????????admin {?????????????? ipaddr = "192.168.0.2"????????????}???????? }??????}?? }}

注:ip就是心跳線的ip
這三個(gè)文件建在/root/cluster下,先建立Cluster Configuration System:
a.#vi /etc/gfs/pool0.cfg
poolname pool0
minor 1 subpools 1
subpool 0 8 1 gfs_data
pooldevice 0 0 /dev/sde1
b.#pool_assemble -a pool0
c.#ccs_tool create /root/cluster /dev/pool/pool0
d.#vi /etc/sysconfig/gfs
CCS_ARCHIVE="/dev/pool/pool0"

再Creating a Pool Volume,就是我們要的共享磁盤啦,
a.#vi /etc/gfs/pool1.cfg
poolname pool1
minor 2 subpools 1
subpool 0 128 1 gfs_data
pooldevice 0 0 /dev/sdd1
b.#pool_assemble -a pool1
c.#gfs_mkfs -p lock_gulm -t Cluster_1:gfs1 -j 8 /dev/pool/pool1
d.#mount -t gfs -o noatime /dev/pool/pool1 /u01
下面是個(gè)GFS的啟動(dòng)腳本,注意real1和real2必須同時(shí)啟動(dòng)lock_gulmd進(jìn)程,第一臺(tái)lock_gulmd
會(huì)成為Server并等Client的lock_gulmd,幾十秒后沒有響應(yīng)會(huì)fail,GFS啟動(dòng)失敗。Redhat建議
GFS盤不要寫進(jìn)/etc/fstab。
#cat gfstart.sh

#!/bin/shdepmod -amodprobe poolmodprobe lock_gulmmodprobe gfssleep 5service iscsi startsleep 20service rawdevices restartpool_assemble -a pool0pool_assemble -a pool1service ccsd startservice lock_gulmd startmount -t gfs /dev/pool/pool1 /s02 -o noatimeservice gfs status
6. Setup Linux LVS
LVS是章文嵩博士發(fā)起和領(lǐng)導(dǎo)的優(yōu)秀的集群解決方案,許多商業(yè)的集群產(chǎn)品,比如RedHat的Piranha,Turbolinux公司的Turbo Cluster等,都是基于LVS的核心代碼的。
我的系統(tǒng)是Redhat AS3U4,就用Piranha了。從rhel-3-u5-rhcs-i386.iso安裝piranha-0.7.10-2.i386.rpm,ipvsadm-1.21-9.ipvs108.i386.rpm (http://distro.ibiblio.org/pub/linux/distributions/caoslinux/centos/3.1/contrib/i386/RPMS/) 裝完后service httpd start & service piranha-gui start,就可以從http://xx.xx.xx.xx:3636管理或設(shè)置了,當(dāng)然了,手工改/etc/sysconfig/ha/lvs.cf也一樣。
#cat /etc/sysconfig/ha/lvs.cf

serial_no = 80primary = 10.3.1.101service = lvsrsh_command = sshbackup_active = 0backup = 0.0.0.0heartbeat = 1heartbeat_port = 1050keepalive = 6deadtime = 18network = natnat_router = 192.168.1.1 eth1:1nat_nmask = 255.255.255.0reservation_conflict_action = preemptdebug_level = NONEvirtual lvs1 {???? active = 1???? address = 10.3.1.254 eth0:1???? vip_nmask = 255.255.255.0???? fwmark = 100???? port = 80???? persistent = 60???? pmask = 255.255.255.255???? send = "GET / HTTP/1.0\r\n\r\n"???? expect = "HTTP"???? load_monitor = ruptime???? scheduler = wlc???? protocol = tcp???? timeout = 6???? reentry = 15???? quiesce_server = 1???? server Real1 {???????? address = 192.168.1.68???????? active = 1???????? weight = 1???? }???? server Real2 {???????? address = 192.168.1.67???????? active = 1???????? weight = 1???? }}virtual lvs2 {???? active = 1???? address = 10.3.1.254 eth0:1???? vip_nmask = 255.255.255.0???? port = 21???? send = "\n"???? use_regex = 0???? load_monitor = ruptime???? scheduler = wlc???? protocol = tcp???? timeout = 6???? reentry = 15???? quiesce_server = 0???? server ftp1 {???????? address = 192.168.1.68???????? active = 1???????? weight = 1???? }???? server ftp2 {???????? address = 192.168.1.67???????? active = 1???????? weight = 1???? }}

設(shè)置完后service pulse start, 別忘了把相關(guān)的client加進(jìn)/etc/hosts
#iptables -t mangle -A PREROUTING -p tcp -d 10.3.1.254/32 --dport 80 -j MARK --set-mark 100
#iptables -t mangle -A PREROUTING -p tcp -d 10.3.1.254/32 --dport 443 -j MARK --set-mark 100
#iptables -A POSTROUTING -t nat -p tcp -s 10.3.1.0/24 --sport 20 -j MASQUERADE
運(yùn)行以上三行命令并存入/etc/rc.d/rc.local,用ipvsadm看狀態(tài):
#ipvsadm

IP Virtual Server version 1.0.8 (size=65536)Prot LocalAddress:Port Scheduler Flags??-> RemoteAddress:Port?????????? Forward Weight ActiveConn InActConnTCP??10.3.1.254:ftp wlc??-> cluster2:ftp??????????????????Masq????1??????0??????????0??-> cluster1:ftp??????????????????Masq????1??????0??????????0FWM??100 wlc persistent 60??-> cluster1:0????????????????????Masq????1??????0??????????0??-> cluster2:0????????????????????Masq????1??????0??????????0

注意:a.Firewall Mark可以不要,我反正是加了,文檔說有https的話加上,值我選了100,
b.Virtual IP別加進(jìn)/etc/hosts,我上過當(dāng),80端口時(shí)有時(shí)無的,
c.eth0:1,eth1:1是piranha產(chǎn)生的,別自己手工設(shè)置,我干過這畫蛇添足的事,網(wǎng)上有
些帖子沒說清,最后是看Redhat的文檔才弄清楚的。
d.The LVS router can monitor the load on the various real servers by using
either rup or ruptime. If you select rup from the drop-down menu, each real
server must run the rstatd service. If you select ruptime, each real server
must run the rwhod service.Redhat的原話,就是如選rup的監(jiān)控模式real server上
都要運(yùn)行rstatd進(jìn)程,如選ruptime就要運(yùn)行rwhod進(jìn)程。
e.Real Server同Router相聯(lián)的網(wǎng)卡的Gateway必須是Router的那塊網(wǎng)卡的VIP,舉本例:
Router的eth1同兩個(gè)real server的eth0相聯(lián),如VIP eth1:1=192.168.1.1,則real
server 的eth0的Gateway=192.168.1.1

echo "1" > /proc/sys/net/ipv4/ip_forward
q:在兩臺(tái)real server上分別
#vi /usr/local/tomcat/conf/tomcat-users.xml
加下面一行,允許頁(yè)面管理:

r:在兩臺(tái)real server上分別
#service httpd restart
#/usr/local/tomcat/bin/startup.sh
s:打開http://1092.168.1.68:8080和http://1092.168.1.67:8080,選Tomcat Manager,用
manager/tomcat登錄,虛擬目錄/myjsp和/local應(yīng)該Start了
在兩臺(tái)機(jī)子上分別打開網(wǎng)頁(yè)http://10.3.1.254,選WEB Local,可以看到一臺(tái)顯示:
"test page on real server 1",另一臺(tái)為"test page on real server 2",同時(shí)在Router上
ipvsadm可以看到每個(gè)real server的聯(lián)接數(shù)
8.設(shè)置FTP服務(wù)
#vi /etc/vsftpd/vsftp.conf,在兩臺(tái)real server上分別加入以下幾行:

anon_root=/u01/ftprootlocal_root=/u01/ftprootsetproctitle_enable=YES

#service vsftpd start
現(xiàn)在LVM+GFS+ISCSI+TOMCAT就設(shè)置好了,我們可以用Apache Jmeter來測(cè)試LVM的性能,兩臺(tái)機(jī)子上分別運(yùn)行jmeter,都指向10.3.1.254/myjsp/testoracle.jsp,各200個(gè)threads同時(shí)運(yùn)行,在Router上用ipvsadm可以監(jiān)控,Oracle Server的性能可要好,否則大量的http進(jìn)程會(huì)hang在real server上,ipvsadm也會(huì)顯示有個(gè)real server失去了。測(cè)試時(shí)real server的CPU idle會(huì)降到70%,而Router的CPU idle幾乎不動(dòng)。
??????????????????????????????????????????????????????文章引用自:大行的BLOG

集群LVS+GFS+ISCSI+TOMCAT


更多文章、技術(shù)交流、商務(wù)合作、聯(lián)系博主

微信掃碼或搜索:z360901061

微信掃一掃加我為好友

QQ號(hào)聯(lián)系: 360901061

您的支持是博主寫作最大的動(dòng)力,如果您喜歡我的文章,感覺我的文章對(duì)您有幫助,請(qǐng)用微信掃描下面二維碼支持博主2元、5元、10元、20元等您想捐的金額吧,狠狠點(diǎn)擊下面給點(diǎn)支持吧,站長(zhǎng)非常感激您!手機(jī)微信長(zhǎng)按不能支付解決辦法:請(qǐng)將微信支付二維碼保存到相冊(cè),切換到微信,然后點(diǎn)擊微信右上角掃一掃功能,選擇支付二維碼完成支付。

【本文對(duì)您有幫助就好】

您的支持是博主寫作最大的動(dòng)力,如果您喜歡我的文章,感覺我的文章對(duì)您有幫助,請(qǐng)用微信掃描上面二維碼支持博主2元、5元、10元、自定義金額等您想捐的金額吧,站長(zhǎng)會(huì)非常 感謝您的哦!!!

發(fā)表我的評(píng)論
最新評(píng)論 總共0條評(píng)論
主站蜘蛛池模板: 久久手机在线视频 | 日本亚洲欧美国产日韩ay高清 | 成人毛片免费在线观看 | 欧美精 | 日韩精品一区二区三区高清 | 80s成年女人毛片免费观看观看 | 婷婷成人综合 | 日韩欧美天堂 | 欧美在线 | 亚洲 | 欧洲自拍偷拍 | 偷偷狠狠的日日日日 | 日本一级大黄毛片免费基地 | 久久99国产乱子伦精品免 | 爱操视频在线观看 | 中文字幕亚洲专区 | 一区二区三区不卡免费视频97 | 色综合久久88中文字幕 | 中文字幕无线码中文字幕免费 | 国产亚洲一区二区三区啪 | 一级黄色毛片子 | 国产乱人伦偷精品视频不卡 | 久久青青草原精品国产麻豆 | 亚洲欧美在线综合一区二区三区 | 亚洲免费中文字幕 | 日本免费成人网 | 一区二区三区在线 | 网站 | 一本大道久久a久久综合 | 性生大片一级毛片免费观看 | 国产毛片久久国产 | 精品无人区乱码一区2区3区 | 日韩a无吗一区二区三区 | 欧美成人毛片一级在线 | 色久综合在线 | 夜夜爽毛片 | 六月婷婷啪啪 | 欧美日韩在大午夜爽爽影院 | 久久久噜噜噜久久老司机 | 国产精品青草久久福利不卡 | 正在播放亚洲 | 久久456| 色视频在线免费看 |