The best way to illustrate how the rewrite process is done internally is to look at an example and see which terms are used instead of the original query term. We will index three documents to our library_it index by using the following commands:

curl -XPOST 'localhost:9200/library_it/book/1' -d '{"title": "Solr 4 Cookbook"}'
curl -XPOST 'localhost:9200/library_it/book/2' -d '{"title": "Solr 3.1 Cookbook"}'
curl -XPOST 'localhost:9200/library_it/book/3' -d '{"title": "Mastering Elasticsearch"}'

What we would like is to find all the documents that start with the letter s. Simple as that, we run the following query against our library_it index:

curl -XGET 'localhost:9200/library_it/_search?pretty' -d '{
 "query" : {
  "prefix" : {
   "title" : {
    "prefix" : "s",
    "rewrite" : "constant_score_boolean"
   }
  }
 }
}'

We've used a simple prefix query; we've said that we would like to find all the documents with the letter s in the title field. We've also used the rewrite property to specify the query rewrite method, but let's skip it for now as we will discuss the possible values of this parameter in the later part of this section.

As the response to the previous query, we get the following:

{
  "took" : 13,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 2,
    "max_score" : 1.0,
    "hits" : [ {
      "_index" : "library_it",
      "_type" : "book",
      "_id" : "2",
      "_score" : 1.0,
      "_source" : {
        "title" : "Solr 3.1 Cookbook"
      }
    }, {
      "_index" : "library_it",
      "_type" : "book",
      "_id" : "1",
      "_score" : 1.0,
      "_source" : {
        "title" : "Solr 4 Cookbook"
      }
    } ]
  }
}

As you can see, in response we got the two documents that had the contents of the title field starting with the desired character. We didn't specify the mappings explicitly, so we relied on Elasticsearch's ability to choose the mapping type for us. As we already know, for the text field, Elasticsearch uses the default analyzer. This means that the terms in our documents will be lowercased and, because of that, we used the lowercased letter in our prefix query (remember that the prefix query is not analyzed).

Now let's take a step back and look at Apache Lucene again. If you recall what Lucene inverted index is built from, you can tell that it contains a term, count, and document pointer (if you don't recall, refer to the Full text searching section in Chapter 1, Getting Started with Elasticsearch Cluster). So, let's see how the simplified view of the index may look for the preceding data we've put to the library_it index:

What you see in the column with the Term text is quite important. If you look at Elasticsearch and Apache Lucene internals, you can see that our prefix query was rewritten to the following Lucene query:

ConstantScore(title:solr)

We can check the portions of the rewrite using the Elasticsearch API. First of all, we can use the Explain API by running the following command:

curl -XGET 'localhost:9200/library_it/book/1/_explain?pretty' -d '{
 "query" : {
  "prefix" : {
   "title" : {
    "prefix" : "s",
    "rewrite" : "constant_score_boolean"
   }
  }
 }
}'

The result will be as follows:

{
  "_index" : "library_it",
  "_type" : "book",
  "_id" : "1",
  "matched" : true,
  "explanation" : {
    "value" : 1.0,
    "description" : "sum of:",
    "details" : [ {
      "value" : 1.0,
      "description" : "ConstantScore(title:solr), product of:",
      "details" : [ {
        "value" : 1.0,
        "description" : "boost",
        "details" : [ ]
      }, {
        "value" : 1.0,
        "description" : "queryNorm",
        "details" : [ ]
      } ]
    }, {
      "value" : 0.0,
      "description" : "match on required clause, product of:",
      "details" : [ {
        "value" : 0.0,
        "description" : "# clause",
        "details" : [ ]
      }, {
        "value" : 1.0,
        "description" : "_type:book, product of:",
        "details" : [ {
          "value" : 1.0,
          "description" : "boost",
          "details" : [ ]
        }, {
          "value" : 1.0,
          "description" : "queryNorm",
          "details" : [ ]
        } ]
      } ]
    } ]
  }
}

We can see that Elasticsearch used a constant score query with the term solr against the title field.

We can control how the queries are rewritten internally. To do that, we place the rewrite parameter inside the JSON object responsible for the actual query. For example:

curl -XGET 'localhost:9200/library/book/_search?pretty' -d '{
   "query" : {
    "prefix" : {
      "title" : "s",
      "rewrite" : "constant_score_boolean"
    }
  }
}'

The rewrite property can take the following values:

For example, if we would like our example query to use top_terms_N with N equal to 2, our query would look like this:

curl -XGET 'localhost:9200/library/book/_search?pretty' -d '{
  "query" : {
    "prefix" : {
     "title" : {
      "prefix" :"s",
      "rewrite" : "top_terms_2"
     }
    }
  }
}'

If you look at the results returned by Elasticsearch, you'll notice that, unlike our initial query, the documents were given a score different than the default 1.0:

{
  "took" : 4,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 0.15342641,
    "hits" : [ {
      "_index" : "library",
      "_type" : "book",
      "_id" : "3",
      "_score" : 0.15342641,
      "_source" : {
        "title" : "The Complete Sherlock Holmes",
        "author" : "Arthur Conan Doyle",
        "year" : 1936,
        "characters" : [ "Sherlock Holmes", "Dr. Watson", "G. Lestrade" ],
        "tags" : [ ],
        "copies" : 0,
        "available" : false,
        "section" : 12
      }
    } ]
  }
}

The score is different than the default 1.0 because we've used the top_terms_N rewrite type and this type of query rewrite keeps the score for N top scoring terms.

Before we finish the Query rewrite section of this chapter, we should ask ourselves one last question: when to use which rewrite type? The answer to this question greatly depends on your use case, but, to summarize, if you can live with lower precision and relevancy (but higher performance), you can go for the top N rewrite method. If you need high precision and thus more relevant queries (but lower performance), choose the Boolean approach.