In the Starting multiple instances as part of a replica set recipe in Chapter 1, Installing and Starting the MongoDB Server, we saw how to set up a simple replica set and what the purpose of a replica set is. We also have a good deal of explanation in Appendix, Concepts for Reference, on what write concern is and why it is used. What we saw about write concerns is that they offer a minimum level guarantee for a certain write operation. However, with the concept of tags and write concerns, we can define a variety of rules and conditions that must be satisfied before a write operation is deemed successful and a response is sent to the user.
Consider some common use cases:
- An application wants a write operation to be propagated to at least one server in each of its data centers. This ensures that, in the event of a data center shutdown, other data centers will have the data that was written by the application.
- If there aren't multiple data centers, at least one member of a replica set is kept on a different rack. For instance, if the rack's power supply goes down, the replica set will still be available (not necessarily for writes) as at least one member is running on a different rack. In such scenarios, we would want the write to be propagated to at least two racks before responding to the client with a successful write.
- It is possible that a reporting application queries a group of secondary instances of a replica set to generate some reports regularly (such a secondary might be configured to never become primary). After each write, we want to ensure that the write operation is replicated to at least one reporting replica member, before acknowledging the write as successful.
The preceding use cases are a few of the common use cases that arise and are not addressed using simple write concerns that we have seen earlier. We need a different mechanism to cater to these requirements; replica sets with tags are what we need.
Obviously the next question is, What exactly are tags? Let us take an example of a blog. Various posts in the blog have different tags attached to them. These tags allow us to easily search, group, and relate posts together. Tags are user-defined texts with some meaning attached to it. If we draw an analogy between a blog post and the replica set members, just as we attach tags to a post, we can attach tags to each replica set member. For example, in a multi-data center scenario with two replica set members in data center 1 (dc1) and one member in data center 2 (dc2), we can have the following tags assigned to the members. The name of the key and the value assigned to the tag are arbitrary, and they are chosen during the designing of the application.
You may even choose to assign any tags, for example, to the administrator who set up the server, if you really find it useful to address your use case.
|
Replica set member |
Tag |
|---|---|
|
Replica set member 1 |
|
|
Replica set member 2 |
|
|
Replica set member 3 |
|
This is good enough to lay the foundation of what replica set tags are. In this recipe, we will see how to assign tags to replica set members and, more importantly, how to make use of them to address some of the sample use cases we saw earlier.
Refer to the Starting multiple instances as part of a replica set recipe in Chapter 1, Installing and Starting the MongoDB Server, for the prerequisites and to know about replica set basics. Go ahead and set up a simple three-node replica set on your computer, as mentioned in the recipe. Open a shell and connect to the primary member of the replica set.
If you need to know about write concerns, refer to the overview of write concerns Appendix, Concepts for Reference.
For the purpose of inserting into the database, we will use Python, as it gives us an interactive interface such as the Mongo shell. Refer to the Installing PyMongo recipe in Chapter 3, Programming Language Drivers, for steps on how to install PyMongo. The Mongo shell would have been the most ideal candidate for the demonstration of the insert operations, but there are certain limitations around the usage of the shell with our custom write concern. Technically, any programming language with the write concerns mentioned in the recipe for insert operations will work fine.
- With the replica set started, we will add tags to it and reconfigure using the following commands that are executed from the Mongo shell:
> var conf = rs.conf() > conf[0].members.tags = {'datacentre': 'dc1', 'rack': 'rack-dc1-1'} > conf[1].members.tags = {'datacentre': 'dc1', 'rack': 'rack-dc1-2'} > conf[2].members.priority = 0 > conf[2].members.tags = {'datacentre': 'dc2', 'rack': 'rack-dc2-1'}
- With the replica set tags set (note that we have not yet reconfigured the replica set), we need to define some custom write concerns. First, we define one that will ensure that the data gets replicated at least to one server in each data center. Execute the following commands in the Mongo shell again:
> conf.settings = {'getLastErrorModes' : {'MultiDC':{datacentre : 2}}} > rs.reconfig(conf)
- Start the Python shell and execute the following commands:
>>> import pymongo >>> client = pymongo.MongoReplicaSetClient('localhost:27000,localhost:27001', replicaSet='replSetTest') >>> db = client.test
- We will now execute the following
insertquery:>>> db.multiDCTest.insert({'i':1}, w='MultiDC', wtimeout=5000) - The preceding
insertquery goes through successfully, andObjectIdwill be printed out. You may query the collection to confirm from either the Mongo shell or the Python shell. - As our primary is one of the servers in data center 1, we will now stop the server listening to port
27002, which is the one with priority0and tagged to be in a different data center. - Once the server is stopped (you may confirm using the
rs.status()helper function from the Mongo shell), execute the followinginsertquery again; this insert should throw an error for timeout:>>> db.multiDCTest.insert({'i':2}, w='MultiDC', wtimeout=5000) - Restart the stopped MongoDB server.
- Similarly, we can achieve rack awareness by ensuring that the write propagates at least two racks (in any data center) by defining a new configuration from the Mongo shell as follows
{'MultiRack':{rack : 2}} - The settings value of the
confobject will then be as follows. Once set, reconfigure the replica set again usingrs.reconfig(conf)from the Mongo shell as follows:{ 'getLastErrorModes' : { 'MultiDC':{datacentre : 2}, 'MultiRack':{rack : 2} } }
We saw
WriteConcernused with replica set tags to achieve functionality such as data center and rack awareness. Let us see how we can use replica set tags with read operations. - We will see how to make use of replica set tags with read preference. Let us reconfigure the set by adding one more tag to mark a secondary member that will be used to execute some hourly stats reporting.
- Execute the following steps to reconfigure the set from the Mongo shell:
> var conf = rs.conf() > conf.members[2].tags.type = 'reports' > rs.reconfig(conf)
- This will configure the same member with priority
0and1in a different data center with an additional tag calledtypewith the valuereports. - We now go back to the Python shell and execute the following commands:
>>> curs = db.multiDCTest.find(read_preference=pymongo.ReadPreference.SECONDARY, tag_sets=[{'type':'reports'}]) >>> curs.next()
- The preceding execution should show us one document from the collection (as we had inserted data in this test collection in the previous steps).
- Stop the instance that we tagged for reporting, that is, the server listening to connections on port
27002, and execute the following command on the Python shell again:>>> curs = db.multiDCTest.find(read_preference=pymongo.ReadPreference.SECONDARY, tag_sets=[{'type':'reports'}]) >>> curs.next()
This time around, the execution should fail and state that no secondary was found with the required tag sets.
In this recipe, we did a lot of operations on tagged replica sets and saw how they can affect write operations using WriteConcern and read operations using ReadPreference. Let us look at them in some detail now.
We set up a replica set that was up and running, which we reconfigured to add tags. We tagged the first two servers in data center 1 and in different racks (with the servers running and listening to ports 27000 and 27001 for client connections), and the third one in data center 2 (with the server listening to port 27002 for client connections). We also ensured that the member in data center 2 doesn't become a primary by setting its priority to 0.
Our first objective is to ensure that write operations to the replica set get replicated to at least one member in the two data centers. To ensure this, we define a write concern as follows:
{'MultiDC':{datacentre : 2}}
Here, we first define the name of the write concern as MultiDC. The value, which is a JSON object, has one key with the name datacenter, which is the same as the key used for the tag we attached to the replica set, and the value is the number 2, which will be looked at as the number of distinct values of the given tag that should acknowledge the write before it is deemed successful.
For instance, in our case, when the write comes to server 1 in data center 1, the number of distinct values of the datacentre tag is 1. If the write operation gets replicated to the second server, the number still stays 1, as the value of the datacentre tag is the same as the first member. It is only when the third server acknowledges the write operation that the write satisfies the defined condition of replicating the write to two distinct values of the datacentre tag in the replica set. Note that the value can only be a number and can not have something such as {datacentre : 'dc1'}. This definition is invalid and an error will be thrown while reconfiguring the replica set.
However, we need to register this write concern somewhere with the server. This is done in the final step of the configuration by setting the settings value in configuration JSON. The value to set is getLastErrorModes. The value of getLastErrorModes is a JSON document with all possible write concerns defined in it. We later define one more write concern for writes propagated to at least two racks. This is conceptually in line with the MultiDC write concern and thus, we will not be discussing it in detail here. After setting all the required tags and settings, we reconfigure the replica set for the changes to take effect.
Once reconfigured, we perform some write operations using the MultiDC write concern. When two members in two distinct data centers are available, the write goes through successfully. However, when the server in the second data center goes down, the write operation times out and throws an exception to the client initiating the write. This demonstrates that the write operation will succeed or fail as per how we intended.
We just saw how these custom tags can be used to address some interesting use cases that are not supported by the product implicitly, as far as write operations are concerned. Similar to write operations, read operations can take full advantage of these tags to address some use cases, such as reading from a fixed set of secondary members that are tagged with a particular value.
We added another custom tag annotating a member to be used for reporting purposes. We then fired a query operation with the read preference to query a secondary and provided the tag sets that should be looked for before considering the member as a candidate for a read operation. Remember that when using a primary as the read preference, we cannot use tags, and that is the reason we explicitly specified the value of read_preference to SECONDARY.