Sharding cluster介紹
這是一種可以水平擴展的模式,在數(shù)據(jù)量很大時特給力,實際大規(guī)模應用一般會采用這種架構去構建monodb系統(tǒng)。
?要構建一個 MongoDB Sharding Cluster,需要三種角色:
Shard Server: mongod 實例,用于存儲實際的數(shù)據(jù)塊,實際生產(chǎn)環(huán)境中一個shard server角色可由幾臺機器組個一個relica set承擔,防止主機單點故障
Config Server: mongod 實例,存儲了整個 Cluster Metadata,其中包括 chunk 信息。
Route Server: mongos 實例,前端路由,客戶端由此接入,且讓整個集群看上去像單一數(shù)據(jù)庫,前端應用可以透明使用。
Sharding架構圖:
本例實際環(huán)境架構
本例架構示例圖:
- 分別在3臺機器運行一個mongod實例(稱為mongod shard11,mongod shard12,mongod shard13)組織replica set1,作為cluster的shard1
- 分別在3臺機器運行一個mongod實例(稱為mongod shard21,mongod shard22,mongod shard23)組織replica set2,作為cluster的shard2
- 每臺機器運行一個mongod實例,作為3個config server
- 每臺機器運行一個mongs進程,用于客戶端連接
主機 ? | IP | 端口信息 |
Server1 | 10.1.1.1 |
mongod shard11:27017
mongod shard12:27018 mongod config1:20000 mongs1:30000 |
Server2 | 10.1.1.2 |
mongod shard12:27017
mongod shard22:27018 mongod config2:20000 mongs2:30000 |
Server3 | 10.1.1.3 |
mongod shard13:27017
mongod shard23:27018 mongod config3:20000 mongs3:30000 |
軟件準備
軟件準備
1.?創(chuàng)建用戶
groupadd -g 20001 mongodb
useradd -u 20001 -g mongodb mongodb
passwd mongodb
2.?安裝monodb軟件
su – mongodb
tar zxvf mongodb-linux-x86_64-1.6.2.tar
安裝好后,目錄結構如下:
$ tree mongodb-linux-x86_64-1.6.2
mongodb-linux-x86_64-1.6.2
|– GNU-AGPL-3.0
|– README
|– THIRD-PARTY-NOTICES
`– bin
??? |– bsondump
??? |– mongo
??? |– mongod
??? |– mongodump
??? |– mongoexport
??? |– mongofiles
??? |– mongoimport
??? |– mongorestore
??? |– mongos
??? |– mongosniff
??? `– mongostat
1 directory, 14 files
3.?創(chuàng)建數(shù)據(jù)目錄
根據(jù)本例sharding架構圖所示,在各臺sever上創(chuàng)建shard數(shù)據(jù)文件目錄
Server1:
su – monodb
cd /home/monodb
mkdir -p data/shard11
mkdir -p data/shard21
Server2:
su – monodb
cd /home/monodb
mkdir -p data/shard11
mkdir -p data/shard22
Server3:
su – monodb
cd /home/monodb
mkdir -p data/shard13
mkdir -p data/shard23
配置relica sets
1.?配置shard1所用到的replica sets:
Server1:
cd /home/mongodb/mongodb-linux-x86_64-1.6.2/bin
./mongod –shardsvr –replSet shard1 –port 27017 –dbpath /home/mongodb/data/shard11 –oplogSize 100 –logpath /home/mongodb/data/shard11.log –logappend –fork
Server2:
cd /home/mongodb/mongodb-linux-x86_64-1.6.2/bin
./mongod –shardsvr –replSet shard1 –port 27017 –dbpath /home/mongodb/data/shard12 –oplogSize 100 –logpath /home/mongodb/data/shard12.log –logappend –fork
Server3:
cd /home/mongodb/mongodb-linux-x86_64-1.6.2/bin
./mongod –shardsvr –replSet shard1 –port 27017 –dbpath /home/mongodb/data/shard13 –oplogSize 100 –logpath /home/mongodb/data/shard13.log –logappend –fork
初始化replica set
用mongo連接其中一個mongod,執(zhí)行:
> config = {_id: ‘shard1′, members: [
????????????????????????? {_id: 0, host: '10.1.1.1:27017'},
????????????????????????? {_id: 1, host: '10.1.1.2:27017'},
????????????????????????? {_id: 2, host: '10.1.1.3:27017'}]
?????????? }
> rs.initiate(config);
同樣方法,配置shard2用到的replica sets:
server1:
cd /home/mongodb/mongodb-linux-x86_64-1.6.2/bin
./mongod –shardsvr –replSet shard2 –port 27018 –dbpath /home/mongodb/data/shard21 –oplogSize 100 –logpath /home/mongodb/data/shard21.log –logappend –fork
server2:
cd /home/mongodb/mongodb-linux-x86_64-1.6.2/bin
./mongod –shardsvr –replSet shard2 –port 27018 –dbpath /home/mongodb/data/shard22 –oplogSize 100 –logpath /home/mongodb/data/shard22.log –logappend –fork
server3:
cd /home/mongodb/mongodb-linux-x86_64-1.6.2/bin
./mongod –shardsvr –replSet shard2 –port 27018 –dbpath /home/mongodb/data/shard23 –oplogSize 100 –logpath /home/mongodb/data/shard23.log –logappend –fork
初始化replica set
用mongo連接其中一個mongod,執(zhí)行:
> config = {_id: ‘shard2′, members: [
????????????????????????? {_id: 0, host: '10.1.1.1:27018'},
????????????????????????? {_id: 1, host: '10.1.1.2:27018'},
????????????????????????? {_id: 2, host: '10.1.1.3:27018'}]
?????????? }
> rs.initiate(config);
到此就配置好了二個replica sets,也就是準備好了二個shards
配置三臺config server
Server1:
mkdir -p /home/mongodb/data/config
./mongod –configsvr –dbpath /home/mongodb/data/config –port 20000 –logpath /home/mongodb/data/config.log –logappend –fork?? #config server也需要dbpath
?
Server2:
mkdir -p /home/mongodb/data/config
./mongod –configsvr –dbpath /home/mongodb/data/config –port 20000 –logpath /home/mongodb/data/config.log –logappend –fork
Server3:
mkdir -p /home/mongodb/data/config
./mongod –configsvr –dbpath /home/mongodb/data/config –port 20000 –logpath /home/mongodb/data/config.log –logappend –fork
配置mongs
在server1,server2,server3上分別執(zhí)行:
./mongos –configdb 10.1.1.1:20000,10.1.1.2:20000,10.1.1.3:20000 –port 30000 –chunkSize 5 –logpath /home/mongodb/data/mongos.log –logappend –fork?
#mongs不需要dbpath
Configuring the Shard Cluster
連接到其中一個mongos進程,并切換到admin數(shù)據(jù)庫做以下配置
1.?連接到mongs,并切換到admin
./mongo 10.1.1.1:30000/admin
>db
Admin
2.?加入shards
如里shard是單臺服務器,用>db.runCommand( { addshard : “<serverhostname>[:<port>]” } )這樣的命令加入,如果shard是replica sets,用replicaSetName/<serverhostname>[:port][,serverhostname2[:port],…]這樣的格式表示,例如本例執(zhí)行:
>db.runCommand( { addshard : “shard1/10.1.1.1:27017,10.1.1.2:27017,10.1.1.3:27017″,name:”s1″,maxsize:20480} );
>db.runCommand( { addshard : “shard2/10.1.1.1:27018,10.1.1.2:27018,10.1.1.3:27018″,name:”s2″,maxsize:20480} );
注意:在添加第二個shard時,出現(xiàn)error:test database 已經(jīng)存在的錯誤,這里用mongo命令連接到第二個replica set,用db.dropDatabase()命令把test數(shù)據(jù)庫給刪除然后就可加入
3.?可選參數(shù)
Name:用于指定每個shard的名字,不指定的話系統(tǒng)將自動分配
maxSize:指定各個shard可使用的最大磁盤空間,單位megabytes
4.?Listing shards
>db.runCommand( { listshards : 1 } )
如果列出了以上二個你加的shards,表示shards已經(jīng)配置成功
5.?激活數(shù)據(jù)庫分片
命令:
> db.runCommand( { enablesharding : “<dbname>” } );
通過執(zhí)行以上命令,可以讓數(shù)據(jù)庫跨shard,如果不執(zhí)行這步,數(shù)據(jù)庫只會存放在一個shard,一旦激活數(shù)據(jù)庫分片,數(shù)據(jù)庫中不同的collection將被存放在不同的shard上,但一個collection仍舊存放在同一個shard上,要使單個collection也分片,還需單獨對collection作些操作
Collecton分片
要使單個collection也分片存儲,需要給collection指定一個分片key,通過以下命令操作:
> db.runCommand( { shardcollection : “<namespace>”,key : <shardkeypatternobject> });
注:
?a.?分片的collection系統(tǒng)會自動創(chuàng)建一個索引(也可用戶提前創(chuàng)建好)
?b. 分片的collection只能有一個在分片key上的唯一索引,其它唯一索引不被允許
One note: a sharded collection can have only one unique index, which must exist on the shard key. No other unique indexes can exist on the collection.
分片collection例子
>db.runCommand( { shardcollection : “test.c1″,key : {id: 1} } )
>for (var i = 1; i <= 200003; i++) db.c1.save({id:i,value1:”1234567890″,value2:”1234567890″,value3:”1234567890″,value4:”1234567890″});
> db.c1.stats()
{
??????? “sharded” : true,
??????? “ns” : “test.c1″,
??????? “count” : 200003,
??????? “size” : 25600384,
??????? “avgObjSize” : 128,
??????? “storageSize” : 44509696,
??????? “nindexes” : 2,
??????? “nchunks” : 15,
??????? “shards” : {
??????????????? “s1″ : {
??????????????????????? “ns” : “test.c1″,
??????????????????????? “count” : 141941,
??????????????????????? “size” : 18168448,
??????????????????????? “avgObjSize” : 128,
??????????????????????? “storageSize” : 33327616,
??????????????????????? “numExtents” : 8,
??????????????????????? “nindexes” : 2,
??????????????????????? “l(fā)astExtentSize” : 12079360,
??????????????????????? “paddingFactor” : 1,
??????????????????????? “flags” : 1,
??????????????????????? “totalIndexSize” : 11157504,
??????????????????????? “indexSizes” : {
??????????????????????????????? “_id_” : 5898240,
??????????????????????????????? “id_1″ : 5259264
??????????????????????? },
??????????????????????? “ok” : 1
??????????????? },
??????????????? “s2″ : {
??????????????????????? “ns” : “test.c1″,
??????????????????????? “count” : 58062,
??????????????????????? “size” : 7431936,
??????????????????????? “avgObjSize” : 128,
??????????????????????? “storageSize” : 11182080,
??????????????????????? “numExtents” : 6,
??????????????????????? “nindexes” : 2,
??????????????????????? “l(fā)astExtentSize” : 8388608,
??????????????????????? “paddingFactor” : 1,
??????????????????????? “flags” : 1,
??????????????????????? “totalIndexSize” : 4579328,
??????????????????????? “indexSizes” : {
??????????????????????????????? “_id_” : 2416640,
??????????????????????????????? “id_1″ : 2162688
??????????????????????? },
??????????????????????? “ok” : 1
??????????????? }
??????? },
??????? “ok” : 1
}
?
[ 轉(zhuǎn)自 : http://www.taobaodba.com/html/525_525.html ]
?
更多文章、技術交流、商務合作、聯(lián)系博主
微信掃碼或搜索:z360901061

微信掃一掃加我為好友
QQ號聯(lián)系: 360901061
您的支持是博主寫作最大的動力,如果您喜歡我的文章,感覺我的文章對您有幫助,請用微信掃描下面二維碼支持博主2元、5元、10元、20元等您想捐的金額吧,狠狠點擊下面給點支持吧,站長非常感激您!手機微信長按不能支付解決辦法:請將微信支付二維碼保存到相冊,切換到微信,然后點擊微信右上角掃一掃功能,選擇支付二維碼完成支付。
【本文對您有幫助就好】元
