Apart from all the things we will discuss in this section, there are three major, operating system related things you need to remember: the number of allowed file descriptors, the virtual memory, and avoiding swapping.

Note that the following section contains information for Linux operating systems, but you can also achieve similar options on Microsoft Windows.

bootstrap.mlockall: true

LABEL=cloudimg-rootfs  /   ext4  defaults,discard  0 0
/dev/xvdb swap swap defaults 0 0

sudo swapoff -a

elasticsearch soft nofile 65536
elasticsearch hard nofile 65536

session required pam_limits.so

bin/elasticsearch -Des.max-open-files=true

[2015-12-20 00:22:19,869][INFO ][bootstrap                ] max_open_files [10240]

sysctl -w vm.max_map_count=262144

Before thinking about Elasticsearch configuration related things, we should remember about giving enough memory to Elasticsearch. In general, we shouldn't give more than 50-60 percent of the total available memory to the JVM process running Elasticsearch. We do that because we want to leave memory for the operating system and for the operating system I/O cache. However, we need to remember that the 50-60 percent figure is not always true. You can imagine having nodes with 256GB of RAM and having indices of 30GB in total on such a node. In such circumstances, even assigning more than 60 percent of physical RAM to Elasticsearch would leave plenty of RAM for the operating system. It is also a good idea to set the Xmx and Xms properties to the same values to avoid JVM heap size resizing.

Another thing to remember are the so called compressed oops (http://docs.oracle.com/javase/7/docs/technotes/guides/vm/performance-enhancements-7.html#compressedOop), the ordinary object pointers. Java virtual machine can be told to use them by adding the -XX:+UseCompressedOops switch. This allows Java virtual machine to use less memory to address the objects on the heap. However, this is only true for heap sizes less than or equal to 31GB. Going for a larger heap means no compressed oops and higher memory usage for addressing the objects on the heap.

As we know, by default the field data cache in Elasticsearch is unbounded. This can be very dangerous, especially when you are using aggregations and sorting on many fields that are analysed, because they don't use doc values by default. If those fields are high cardinality ones, then you can run into even more trouble. By trouble we mean running out of memory.

We have two different factors we can tune to be sure that we don't run into out of memory errors. First of all, we can limit the size of the field data cache and we should do that. The second thing is the circuit breaker, which we can easily configure to just throw exceptions instead of loading too much data. Combining these two things together will ensure that we don't run into memory issues.

However, we should also remember that Elasticsearch will evict data from the field data cache if its size is not enough to handle aggregation requests or sorting. This will affect the query performance because loading the field data information is not very efficient and is resource intensive. However, in our opinion, it is better to have our queries slower than having our cluster blown up because of out of memory errors.

The field data cache and caches in general were discussed in the Elasticsearch caches section of Chapter 9, Elasticsearch Cluster in Detail.

Whenever you plan to use sorting, aggregations, or scripting heavily, you should use doc values whenever you can. This will not only save you the memory needed for the field data cache, because of fewer objects produced, it will also make the Java virtual machine work better with lower garbage collector time. Doc values were discussed in the Mappings Configuration section of Chapter 2, Indexing Your Data.

In the Elasticsearch caches section of Chapter 9, Elasticsearch Cluster in Detail, we also discussed. There are a few things we would like to mention. First of all, the more RAM for the indexing buffer, the more documents Elasticsearch will be able to hold in memory. So the more memory we have for indexing, the less often the flush to disk will happen and fewer segments will be created. Because of that, your indexing will be faster. But of course, we don't want Elasticsearch to occupy 100 percent of the available memory. Keep in mind that the RAM buffers are set per shard, so the amount of memory that will be used depends on the number of shards and replicas that are assigned on the given node and on the number of documents you index. You should set the upper limits so your node doesn't blow up when it has multiple shards assigned.

Elasticsearch uses Lucene and we know it by now. The thing with Lucene is that the view of the index is not refreshed when new data is indexed or segments are created. To see the newly indexed data, we need to refresh the index. By default, Elasticsearch does that once every second and the period of refresh is controlled by using the index.refresh_interval property, specified per index. The lower the refresh rate, the sooner the documents will be visible for search operations. However, that also means that Elasticsearch will need to put more resources in to refreshing the index view, meaning that the indexing and searching operations will be slower. The higher the refresh rate, the more time you will have to wait before being able to see the data in the search results, but your indexing and querying will be faster.

We haven't talked about thread pools until now, but we would like to mention them now. Each Elasticsearch node holds several thread pools that control the execution queues for operations such as indexing or querying. Elasticsearch uses several pools to allow control over how the threads are handled and much the memory consumption is allowed for user requests.

There are many thread pools (we can specify the type we are configuring by specifying the type property). However, for performance, the most important are:

For example, if we want to configure the thread pool for indexing operations to have a size of 100 and a queue of 500, we will set the following in the elasticsearch.yml configuration file:

threadpool.index.size: 100
threadpool.index.queue_size: 500

Also remember that the thread pool configuration can be updated using the cluster update API. For example, like this:

curl -XPUT 'localhost:9200/_cluster/settings' -d '{
 "transient" : {
  "threadpool.index.size" : 100,
  "threadpool.index.queue_size" : 500
 }
}'

In general, you don't need to work with the thread pools and their configuration. However, when configuring your cluster, you may want to put more emphasis on indexing or querying and, in such cases, giving more threads or larger queues to the prioritized operation may result in more resources being used for such operations.