Capped collections are fixed size collections where documents are added towards the end of the collection, similar to a queue. As capped collection have a fixed size, older documents are removed if the limit is reached.
They are naturally sorted by the order of the insertion and any retrieval needed on them required ordered by time can be retrieved using the $natural sort order. This makes document retrieval very fast.
The following figure gives a pictorial representation of a capped collection of a size which is good enough to hold up to three documents of equal size (which is too small for any practical use, but good for understanding). As we can see in the image, the collection is similar to a circular queue where the oldest document is replaced by the newly added document should the collection become full. The tailable cursors are special types of cursors that tail the collection similar to a tail command in Unix. These cursors iterate through the collection similar to a normal cursors do, but additionally wait for data to be available in the collection if it is not available. We will see capped collections and tailable cursors in detail in this recipe.
Look at the recipe Installing single node MongoDB recipe in Chapter 1, Installing and Starting the Server and start a single instance of Mongo. That is the only prerequisite for this recipe. Start a MongoDB shell and connect to the started server.
There are two parts to this recipe: in the first part, we will create a capped collection called testCapped and try performing some basic operations on it. Next, we will be creating a tailable cursor on this capped collection.
- Drop the collection if one already exists with this name.
> db.testCapped.drop() - Now create a capped collection as follows. Note the size given here is the size in bytes allocated for the collection and not the number of documents it contains:
> db.createCollection('testCapped', {capped : true, size:100}) - We will now insert 100 documents in the capped collection as follows:
> for(i = 1; i < 100; i++) { db.testCapped.insert({'i':i, val:'Test capped'}) }
- Now query the collection as follows:
> db.testCapped.find() - Try to remove the data from the collection as follows:
> db.testCapped.remove() - We will now create and demonstrate a tailable cursor. It is recommended that you type/copy the following pieces of code into a text editor and keep it handy for execution.
- To insert data in a collection, we will be using the following fragment of code. Execute this piece of code in the shell:
> for(i = 101 ; i < 500 ; i++) { sleep(1000) db.testCapped.insert({'i': i, val :'Test Capped'}) }
- To tail a capped collection, we use the following piece of code:
> var cursor = db.testCapped.find().addOption(DBQuery.Option.tailable).addOption(DBQuery.Option.awaitData) while(cursor.hasNext()) { var next = cursor.next() print('i: ' + next.i + ', value: ' + next.val) }
- Open a shell and connect to the running mongod process. This will be the second shell opened and connected to the server. Copy and paste the code mentioned in step 8 in this shell and execute it.
- Observe how the records inserted are shown as they are inserted into the capped collection.
We will create a capped collection explicitly using the createCollection function. This is the only way a capped collection is created. There are two parameters to the createCollection function. The first one is the name of the collection and the second is a JSON document that contains the two fields, capped and size, which are used to inform the user that the collection is capped or not and the size of the collection in bytes respectively. An additional field max can be provided to specify the maximum number of documents in the collection. The field size is required even if the max field is specified. We then insert and query the documents. When we try to remove the documents from the collection, we would see an error that removal is not permitted from the capped collection. It allows the documents to be deleted only when new documents are added and there isn't space available to accommodate them.
What we see next is a tailable cursor we created. We start two shells and one of them is a normal insertion of documents with an interval of 1 second between subsequent insertions. In the second shell, we create a cursor and iterate through it and print the documents that we get from the cursor onto the shell. The additional options we added to the cursor make the difference though. There are two options added, DBQuery.Option.tailable and DBQuery.Option.awaitData. These options are for instructing that the cursor is tailable, rather than normal, where the last position is marked and we can resume where we left off, and secondly to wait for more data for some time rather than returning immediately when no data is available and when we reach towards the end of the cursor, respectively. The awaitData option can be used with tailable cursors only. The combination of these two options gives us a feel similar to the tail command in Unix filesystem.
For a list of available options, visit the following page: http://docs.mongodb.org/manual/reference/method/cursor.addOption/.