There are a few important things when dealing with search or data analysis. We need the results to be precise, we need them to be relevant, and we need them to be returned as soon as possible. If you are a person responsible for designing queries that are run against Elasticsearch, sooner or later, you will find yourself in a position where you will need to improve the performance of your queries. The reasons can vary from hardware-based problems to bad data architecture to poor query design. When writing this book, the benchmark API was only available in the trunk of Elasticsearch, which means that it was not a part of official Elasticsearch distribution. For now we can either use tools like jMeter or ab (the Apache benchmark is http://httpd.apache.org/docs/2.2/programs/ab.html) or use trunk version of Elasticsearch. Please also note that the functionality we are describing can change with the final release, so keeping an eye on http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/search-benchmark.html is a good idea if you want to use benchmarking functionality.
By default, the benchmarking functionality is disabled. Any attempt to use benchmarking on the Elasticsearch node that is not configured properly will lead to an error similar to the following one:
{
"error" : "BenchmarkNodeMissingException[No available nodes for executing benchmark [benchmark_name]]",
"status" : 503
}This is okay; no one wants to take a risk of running potentially dangerous functionalities on production cluster. During performance testing and benchmarking, you will want to run many complicated and heavy queries, so running such benchmarks on the Elasticsearch cluster that is used by real users doesn't seem like a good idea. It will lead to the slowness of the cluster, and it could result in crashes and a bad user experience. To use benchmarking, you have to inform Elasticsearch which nodes can run the generated queries. Every instance we want to use for benchmarking should be run with the --node.bench option set to true. For example, we could run an Elasticsearch instance like this:
bin/elasticsearch --node.bench true
The other possibility is to add the node.bench property to the elasticsearch.yml file and, of course, set it to true. Whichever way we choose, we are now ready to run our first benchmark.
Elasticsearch provides the _bench REST endpoint, which allows you to define the task to run on benchmarking-enabled nodes in the cluster. Let's look at a simple example to learn how to do that. We will show you something practical; in the Handling filters and why it matters section in Chapter 2, Power User Query DSL, we talked about filtering. We tried to convince you that, in most cases, post filtering is bad. We can now check it ourselves and see whether the queries with post filtering are really slower. The command that allows us to test this looks as follows (we have used the Wikipedia database):
curl -XPUT 'localhost:9200/_bench/?pretty' -d '{ "name": "firstTest", "competitors": [ { "name": "post_filter", "requests": [ { "post_filter": { "term": { "link": "Toyota Corolla" } } }] }, { "name": "filtered", "requests": [ { "query": { "filtered": { "query": { "match_all": {} }, "filter": { "term": { "link": "Toyota Corolla" } } } } }] }] }'
The structure of a request to the _bench REST endpoint is pretty simple. It contains a list of competitors—queries or sets of queries (because each competitor can have more than a single query)—that will be compared to each other by the Elasticsearch benchmarking functionality. Each competitor has its name to allow easier results analysis. Now, let's finally look at the results returned by the preceding request:
{
"status": "COMPLETE",
"errors": [],
"competitors": {
"filtered": {
"summary": {
"nodes": [
"Free Spirit"
],
"total_iterations": 5,
"completed_iterations": 5,
"total_queries": 5000,
"concurrency": 5,
"multiplier": 1000,
"avg_warmup_time": 6,
"statistics": {
"min": 1,
"max": 5,
"mean": 1.9590000000000019,
"qps": 510.4645227156713,
"std_dev": 0.6143244085137575,
"millis_per_hit": 0.0009694501018329939,
"percentile_10": 1,
"percentile_25": 2,
"percentile_50": 2,
"percentile_75": 2,
"percentile_90": 3,
"percentile_99": 4
}
}
},
"post_filter": {
"summary": {
"nodes": [
"Free Spirit"
],
"total_iterations": 5,
"completed_iterations": 5,
"total_queries": 5000,
"concurrency": 5,
"multiplier": 1000,
"avg_warmup_time": 74,
"statistics": {
"min": 66,
"max": 217,
"mean": 120.88000000000022,
"qps": 8.272667107875579,
"std_dev": 18.487886855778815,
"millis_per_hit": 0.05085254582484725,
"percentile_10": 98,
"percentile_25": 109.26595744680851,
"percentile_50": 120.32258064516128,
"percentile_75": 131.3181818181818,
"percentile_90": 143,
"percentile_99": 171.01000000000022
}
}
}
}
}As you can see, the test was successful; Elasticsearch returned an empty errors table. For every test we've run with both post_filter and filtered queries, only a single node named Free Spirit was used for benchmarking. In both cases, the same number of queries was used (5000) with the same number of simultaneous requests (5). Comparing the warm-up time and statistics, you can easily draw conclusions about which query is better. We would like to choose the filtered query; what about you?
Our example was quite simple (actually it was very simple), but it shows you the usefulness of benchmarking. Of course, our initial request didn't use all the configuration options exposed by the Elasticsearch benchmarking API. To summarize all the options, we've prepared a list of the available global options for the _bench REST endpoint:
name: This is the name of the benchmark, making it easy to distinguish multiple benchmarks (refer to the Controlling currently run benchmarks section).competitors: This is the definition of tests that Elasticsearch should perform. It is the array of objects describing each test.num_executor_nodes: This is the maximum number of Elasticsearch nodes that will be used during query tests as a source of queries. It defaults to1.percentiles: This is an array defining percentiles Elasticsearch should compute and return in results with the query execution time. The default value is[10, 25, 50, 75, 90, 99].iteration: This defaults to5and defines the number of repetitions for each competitor that Elasticsearch should perform.concurrency: This is the concurrency for each iteration and it defaults to5, which means that five concurrent threads will be used by Elasticsearch.multiplier: This is the number of repetitions of each query in the given iteration. By default, the query is run1000times.warmup: This informs you that Elasticsearch should perform the warm-up of the query. By default, the warm-up is performed, which means that this value is set totrue.clear_caches: By default, this is set tofalse, which means that before each iteration, Elasticsearch will not clean the caches. We can change this by setting the value totrue. This parameter is connected with a series of parameters saying which cache should or should not be cleared. These additional parameters areclear_caches.filter(the filter cache),clear_caches.field_data(the field data cache),clear_caches.id(the ID cache), andclear_caches.recycler(the recycler cache). In addition, there are two parameters that can take an array of names:clear_caches.fieldsspecifies the names of fields and which cache should be cleared andclear_caches.filter_keysspecifies the names of filter keys to clear. For more information about caches, refer to the Understanding Elasticsearch caching section in Chapter 6, Low-level Index Control.
In addition to the global options, each competitor is an object that can contain the following parameters:
name: Like its equivalent on the root level, this helps distinguish several competitors from each other.requests: This is a table of objects defining queries that should be run within given competitors. Each object is a standard Elasticsearch query that is defined using the query DSL.num_slowest: This is the number of the slowest queries tracked. It defaults to1. If we want Elasticsearch to track and record more than one slow query, we should increase the value of that parameter.search_type: This indicates the type of searches that should be performed. Few of the options arequery_then_fetch,dfs_query_then_fetch, andcount. It defaults toquery_then_fetch.indices: This is an array with indices names to which the queries should be limited.types: This is an array with type names to which the queries should be limited.iteration,concurrency,multiplier,warmup,clear_caches: These parameters override their version defined on the global level.
Depending on the parameters we've used to execute our benchmark, a single benchmarking command containing several queries with thousands of repeats can run for several minutes or even hours. It is very handy to have a possibility to check how the tests run and estimate how long it will take for the benchmark command to end. As you can expect, Elasticsearch provides such information. To get this, the only thing we need to do is run the following command:
curl -XGET 'localhost:9200/_bench?pretty'
The output generated for the preceding command can look as follows (it was taken during the execution of our sample benchmark):
{
"active_benchmarks" : {
"firstTest" : {
"status" : "RUNNING",
"errors" : [ ],
"competitors" : {
"post_filter" : {
"summary" : {
"nodes" : [
"James Proudstar" ],
"total_iterations" : 5,
"completed_iterations" : 3,
"total_queries" : 3000,
"concurrency" : 5,
"multiplier" : 1000,
"avg_warmup_time" : 137.0,
"statistics" : {
"min" : 39,
"max" : 146,
"mean" : 78.95077720207264,
"qps" : 32.81378178835111,
"std_dev" : 17.42543552392229,
"millis_per_hit" : 0.031591310251188054,
"percentile_10" : 59.0,
"percentile_25" : 66.86363636363637,
"percentile_50" : 77.0,
"percentile_75" : 89.22727272727272,
"percentile_90" : 102.0,
"percentile_99" : 124.86000000000013
}
}
}
}
}
}
}Thanks to it, you can see the progress of tests and try to estimate how long you will have to wait for the benchmark to finish and return the results. If you would like to abort the currently running benchmark (for example, it takes too long and you already see that the tested query is not optimal), Elasticsearch has a solution. For example, to abort our benchmark called firstTest, we run a POST request to the _bench/abort REST endpoint, just like this:
curl -XPOST 'localhost:9200/_bench/abort/firstTest?pretty'
The response returned by Elasticsearch will show you a partial result of the test. It is almost the same as what we've seen in the preceding example, except that the status of the benchmark will be set to ABORTED.