Contents

Step by step Mongo DB sharded cluster deployment

05 Mar 2018 0 8  0
Step by step Mongo DB sharded cluster deployment
Step by step Mongo DB sharded cluster deployment

 

Mongo db shard cluster deployment

In this tutorial, we will teach you step by step how to deploy a sharded MongoDB cluster. The Mongo DB version used was 3.2. We will also assume the OS to be Ubuntu. Commands might vary from OS to OS, but the concept should remain the same.

Prerequisites

Basics of Mongo DB sharding.

Architecture

 

http://codingmiles.com/content/images/uploads/2016/02/mongo-db-arch-1024×711.png

In the diagram above you will be able to learn which architecture we are going to be using while deploying the cluster.

The table will explain the servers:

**Machine Type****Components Installed****Description****IP Address****Hostname**
App Server 1Application, MongosThis server will server dual role of app server as well as the mongos server10.10.10.10appserver01
App Server 2Application, MongosThis server will server dual role of app server as well as the mongos server10.10.10.11appserver02
Mongo Config 1Mongo Config ServerUsed as mongodb config server10.10.10.12mongoconfig01
Mongo Config 2Mongo Config ServerUsed as mongodb config server10.10.10.13mongoconfig02
Mongo Config 3Mongo Config ServerUsed as mongodb config server10.10.10.14mongoconfig03
Shard 1 PrimaryMongo DBUsed as primary DB server in shard 110.10.10.15mongosh01db01
Shard 1 SecondaryMongo DBUsed as secondary DB server in shard 110.10.10.16mongosh01db02
Shard 1 SecondaryMongo DBUsed as secondary DB server in shard 110.10.10.17mongosh01db03
Shard 2 PrimaryMongo DBUsed as primary DB server in shard 210.10.10.18mongosh02db01
Shard 2 SecondaryMongo DBUsed as secondary DB server in shard 210.10.10.19mongosh02db02
Shard 2 SecondaryMongo DBUsed as secondary DB server in shard 2

Steps

Begin by logging in as the root user to each server or ensure that you have sudo rights. Every command is made without sudo. Make sure to use sudo if you aren’t a root user.

 

Step 1: Configure hostname of each server

Configure the hostname of every server as per the above table. You could select another hostname, however, ensure that you use the same hostnames in every step. Update the hostnames in the following spots:

 

  1. /etc/hostname
  2. /etc/hosts

Step 2: Install mongo DB on all servers

To install mongo DB on all 11 servers, use the steps mentioned in the link below:

 

 

https://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/

Step 3: Configure Shard 1 replica set

Let’s begin by starting up with a replica set of 3 nodes with 1 primary and 2 secondary.

We are going to name the replicaSet “rs0”.

Step 3.1: Add host entries for members in replica set

 

Modify /etc/hosts and append the entries below:

 

10.10.10.15 mongosh01db01 10.10.10.16 mongosh01db02 10.10.10.17 mongosh01db03

Step 3.2: Edit configuration file for each node in replica set

Modify /etc/mongod.conf and update the following entries:

 

change bindIp to 0.0.0.0
bindIp: 0.0.0.0

Add the configuration for replica set **rs0
replication: replSetName: rs0

 

The overall configuration file should look like the following:

 

storage: dbPath: /var/lib/mongodb journal: enabled: true systemLog: destination: file logAppend: true path: /var/log/mongodb/mongod.log net: port: 27017 bindIp: 0.0.0.0 replication: replSetName: rs0

 

Restart mongo db service after the changes

sudo service mongod restart

 

Step 3.3: Configure the replica set

Follow the steps below to configure the replica set.

 

 

Connect to mongo db using command:
mongo




rs.initiate() rs.add("mongosh01db02") rs.add("mongosh01db03")

 

Confirm the replica set status with rs.status(). It should list one of the servers as primary whilst the others are secondary.

Step 4: Configure Shard 2 replica set

Follow the steps shown in Step 3 with the following servers with the replica set name as rs1:

 

 

mongosh02db01 mongosh02db02 mongosh02db03

Step 5: Configure Mongo config servers

We will have to set up the mongo config servers as a replica set, too.

 

Step 5.1: Add host entries for members in replica set

Config servers should know everything about the servers in the cluster. Append the host entries for all the servers in the /etc/hosts file.

 

 

10.10.10.10 appserver01 10.10.10.11 appserver02 10.10.10.12 mongoconfig01 10.10.10.13 mongoconfig02 10.10.10.14 mongoconfig03 10.10.10.15 mongosh01db01 10.10.10.16 mongosh01db02 10.10.10.17 mongosh01db03 10.10.10.18 mongosh02db01 10.10.10.19 mongosh02db02 10.10.10.20 mongosh02db03

 

 

Step 5.2: Edit configuration file for each node in replica set

Modify /etc/mongod.conf**** then update the following entries.

 

  1. change bindIp to 0.0.0.0
    bindIp: 0.0.0.0
  2. Change port number to 27019
    port: 27019
  3. Add the configuration for replica set with name **configReplSet
    
    replication: replSetName: configReplSet
  4. Configure the server as a config server**sharding: clusterRole: “configsvr”
  5. Your overall configuration file should look like
    storage: dbPath: /var/lib/mongodb journal: enabled: true systemLog: destination: file logAppend: true path: /var/log/mongodb/mongod.log net: port: 27019 bindIp: 0.0.0.0 replication: replSetName: rs0 sharding: clusterRole: "configsvr"
  6. Restart mongo db service after the changes
    sudo service mongod restart

 

Step 5.3: Configure the config server replica set

For any of the config server nodes, follow the steps below in order to configure the replica set.

 

  1. Connect to mongo db using command (note the change in port number):
    mongo mongoconfig01:27019
  2. Add the other config server nodes in the replica set using:
    initiate( { _id: "configReplSet", configsvr: true, members: [ { _id: 0, host: "mongoconfig01:27019" }, { _id: 1, host: "mongoconfig02:27019" }, { _id: 2, host: "mongoconfig03:27019" } ] } )

 

**Check the config replica set status using rs.status(). It should list one of the servers as **primary while other two as secondary.

Step 6: Configure Mongos servers

The steps below will be for both the mongos servers.

 

Step 6.1: Add host entries for members in replica set

Mongos servers need to know everything about the servers in the cluster. Append the host entries for every server in the /etc/hosts file.

 

 

10.10.10.10 appserver01 10.10.10.11 appserver02 10.10.10.12 mongoconfig01 10.10.10.13 mongoconfig02 10.10.10.14 mongoconfig03 10.10.10.15 mongosh01db01 10.10.10.16 mongosh01db02 10.10.10.17 mongosh01db03 10.10.10.18 mongosh02db01 10.10.10.19 mongosh02db02 10.10.10.20 mongosh02db03

 

Step 6.2: Edit configuration file for each mongos

 

Modify /etc/mongod.conf then update the following entries:

 

  1. change bindIp to 0.0.0.0
    bindIp: 0.0.0.0
  2. Delete storage section from the config file
  3. Add the configuration for config servers under sharding
    sharding: configDB:configReplSet/mongoconfig01:27019,mongoconfig02:27019,mongoconfig03:27019
  4. Your overall configuration file should look like
    systemLog: destination: file logAppend: true path: /var/log/mongodb/mongod.log net: port: 27017 bindIp: 0.0.0.0 sharding: configDB: configReplSet/mongoconfig01:27019,mongoconfig02:27019,mongoconfig03:27019
  5. Setup mongos as a service
    cp /etc/init/mongod.conf /etc/init/mongos.conf edit /etc/init/mongos.conf Update DAEMON=/usr/bin/mongod to DAEMON=/usr/bin/mongos Update if [ -f /etc/default/mongod ]; then . /etc/default/mongod; fi to if [ -f /etc/default/mongos ]; then . /etc/default/mongos; fi
  6. Restart mongo db service after the changes (note mongos instead of mongod)
    sudo service mongos restart

 

Step 6.3: Configure the shards

 

Connect to mongos using command:

 

mongo appserver01:27017

 

Add the replica sets as shards. Adding any one of the servers in a replica set will add all the servers.
sh.addShard( "rs0/mongosh01db01:27017") sh.addShard( "rs1/mongosh02db01:27017" )

 

Verify the status with the sh.status() command, which will list the shard servers in the cluster. You could also use the shardMap command to retrieve the complete shard map as follows:

 

use admin; db.runCommand("getShardMap");

Conclusion

The setup of the mongo DB is now finished, feel free to connect the app to the mongos query router with localhost:27017. Reminder that you have to enable sharding for every database and shard the collection you intent to shard.

0 Comments
Add a comment

Leave a Reply