MMS backup is a relatively new offering by MongoDB for real-time incremental backup of your MongoDB instances, replica sets, and shards, and it offers you point-in-time recovery for your instances. The service is available as on-prem (in your data center) or cloud. We will, however, be demonstrating the on-cloud service, which is the only option for the community and basic subscriptions. For more details on the available options, you can refer to the different product offerings by MongoDB at https://www.mongodb.com/products/subscriptions.
The Mongo MMS backup service will work only on Mongo 2.0 and above. We will start a single server that we would backup. MMS backup relies on the oplog for continuous backup, and as oplog is available only in replica sets, the server needs to be started as a replica set. Refer to the Installing PyMongo recipe in Chapter 3, Programming Language Drivers, to know more about how to install Python and the Python client of Mongo, PyMongo.
- If you don't have an MMS account already, then log in to https://mms.mongodb.com/ and sign up for an account. For screenshots, refer to the Signing up for MMS and setting up the MMS monitoring agent recipe.
- Start a single instance of Mongo by replacing the value of the appropriate filesystem path on your machine as follows:
$ mongod --replSet testBackup --smallfiles --oplogSize 50 --dbpath /data/mongo/dbNote that
smallfilesandoplogSizeare options set only for the purpose of testing and they are not to be used in production. - Start a shell, connect to this started instance, and initiate the replica set as follows:
> rs.initiate()The replica set will be up and running in some time.
- Go back to the browser and point to mms.mongodb.com. Add a new host by clicking on the + Add Host button. Select the type as replica set and
hostnameas your hostname and the default port (27017in our case). Refer to the Signing up for MMS and setting up the MMS monitoring agent recipe for the screenshots of the add host process. - Once the host is successfully added, register for MMS backup by clicking on the Backup option on the left-hand side and then on Begin Setup.
- An SMS or Google Authenticator can be used for registration. If a smartphone is available with Android, iOS, or Blackberry OS, Google Authenticator is a good option. For some countries such as India, Google Authenticator is the only option available.
- Assuming Google Authenticator is not configured already and we are planning to use it, we would need the app to be installed on your smartphone. Go to the respective app store of your mobile OS platform and install the Google Authenticator software.
- With the software installed on the phone, come back to the browser. We should see the following screen on selecting Google Authenticator:
- Begin the set up for a new account by scanning the QR code from the Google Authenticator application. If barcode scanning is a problem, you may choose to manually enter the key given on the right-hand side of the screen.
- Once the scanning is completed or the key is entered successfully, your smartphone should show a six-digit number that changes every 30 seconds. Enter that number in the Authentication Code box given on the screen.
Note
It is important not to delete this account in Google Authenticator on your phone, as this would be used in future whenever we wish to change any settings related to backup, such as stopping backup, changing the exclusion list, and literally any operation in MMS backup. The QR code and key would not be visible again once the setup is done. You would have to contact MongoDB support to get the configuration reset.
- Once the authentication is done, the next screen you should see is for the billing address and billing details, such as the card you register. All charges below USD 5 are waived off, so you should be ok to try out a small test instance before being charged.
- Once the credit card details are saved, we move ahead with the setup. We will have to install a backup agent; this is a separate agent from the monitoring agent. Choose the appropriate platform and follow the instructions for its installation. Take note of the location where the configuration files of the agent will be placed.
- A new pop up will contain the instructions/link to the archive/installer for the platform and the steps to install. It should also contain the
apiKey. Take note of that API key, which we will need in the next step. - Once the installation is complete, open the
local.configfile placed in theconfigdirectory of the agent installation (the location that was shown/modified during the installation of the agent) and paste/type in theapiKeynoted down in the previous step. - Once the agent is configured and started, click on the Verify Agent button.
- Once the agent is successfully verified, we should start by adding a host to back up. The dropdown should show us all the replica sets and shards we have added. Select the appropriate one and fill the sync source as the primary instance, as that is the only one we have in our standalone instance. Sync source is only used for the initial sync process. Whenever we have a proper replica set with multiple instances, it is preferable to use a secondary as a sync-process instance.
As the instance is not started with security, leave the DB Username and Password fields blank.
- Click on the Manage excluded namespaces button if you wish to skip a particular database or collection being backed up. If nothing is provided, by default, everything will be backed up. The format for the collection name would be
<databasename>.<collection name>. Alternatively, it could be just the database name, in which case all collections in that database would not be eligible for backup. - Once the details are all ok, click on the Start button. This should complete the setup of the backup process for a replica set on MMS.
The steps are pretty simple and this is all we need to do to set up a server for Mongo MMS backup. One important thing mentioned earlier is that MMS backup uses multifactor authentication for any operation once the backup is set up; the account setup in Google Authenticator for MongoDB should not be deleted. There is no way to recover the original key used to set up the authenticator. You will have to clear the Google Authenticator settings and set up a new key. To do that, click on the Help & Support link at the bottom left of the screen and click on How do I reset my two-factor authentication?.
On clicking the link, a new window will open up, as shown in the following screenshot, which will ask for the username. An e-mail will be sent out to the registered e-mail ID, which allows you to reset the two-factor authentication.
As mentioned, oplog is used to synchronize the current MongoDB instance with the MMS service. However, for the initial sync, an instance's data files are used. Which instance to use is provided by us when we set up the backup of the replica set. As this is a resource-heavy operation, we must preferably use a secondary instance for this on busy systems so as not to add more querying on the primary instance by the MMS backup agent. Once the instance is done with initial synchronization, the oplog of the primary will be used to get data on a continuous basis. The agent does write periodically to a collection called mms.backup in the admin database.
The backup agent for MMS backup is different from the MMS monitoring agent. Though there is no restriction on having them both run on the same machine, you might need to evaluate that before having such a setup in production. A safe bet would be to have them running on separate machines. Never run either of these agents with a mongod or mongos instance on the same box in production. There are a couple of important reasons why it is not recommended to run the agents on the same box as the mongod instances. They are as follows:
- The resource utilization of the agent is dependent on the cluster size it monitors. We don't want the agent to use a lot of resources affecting the performance of the production instance.
- The agent could be monitoring a lot of server instances at one time. As there is only one instance of this agent, we do not want it to go down during the database server maintenance and restart.
The community edition of MongoDB built with SSL or the Enterprise versions, with the SSL option used for communication between the client and the MongoDB server, must perform some additional steps. The first step is to check the My deployment supports SSL for MongoDB connections flag when we set up the replica set for backup (see step 16). Note the checkbox at the bottom of the screenshot; it should be checked. Secondly, open the local.config file for the MMS configuration and look out for the following two properties:
sslTrustedServerCertificates=sslRequireValidServerCertificates=true
The first is a fully-qualified path of the certifying authority's certificate in the PEM format. This certificate will be used to verify the certificate presented by the mongod instance running over SSL. The second property can be set to false if the certificate verification is to be disabled; this is however not a recommended option. As far as the traffic between the backup agent and MMS backup is concerned, data sent from the agent to the MMS service over SSL is secure, irrespective of whether SSL is enabled on your MongoDB instances or not. The data at rest in the data center for the backed up data is not encrypted.
If security is enabled on the mongod instance, a username and password needs to be provided, which will be used by the MMS backup agent. The username and password are provided while setting up backup for the replica set, as seen in step 16.
As the agent needs to read the oplog, possibly all databases for the initial sync and write data to the admin database; the roles expected from the user are readAnyDatabase, clusterAdmin, readWrite on admin and local database, and userAdminAnyDatabase database role in the case of version 2.4 and above. In versions prior to 2.4, we would expect the user to have read access on all the databases and read/write access to admin and local databases.
While setting up a replica set for backup you may get an error such as, Insufficient oplog size: The oplog window must be at least 1 hours over the last 24 hours for all active replica set members. Please increase the oplog. While you may think this is always something to do with oplog size, it is also seen when the replica set has an instance that is in a recovery state. This might feel misleading, so do look out for recovering nodes, if any, in the replica set, while setting up a backup for a replica set. As per the MMS support, it seems too restrictive to not let set up a replica set for backup with some recovering nodes and it might be fixed in future.