This is the only way to allow dynamic scripting if we don't want to enable query dynamic scripting in Elasticsearch. The idea is that every script used by the queries is defined in its own file placed in the config/scripts directory. We will now look at this method of using scripts. Let's create an example file called tag_sort.groovy and let's place it in the config/scripts directory of our Elasticsearch instance (or instances if we run a cluster). The content of the mentioned file should look like this:

_doc.tags.values.size() > 0 ? _doc.tags.values[0] : 'u19999'

After few seconds, Elasticsearch will automatically load a new file. You should see something like the following in the Elasticsearch logs:

[2015-08-30 13:14:33,005][INFO ][script                   ] [Alex Wilder] compiling script file [/Users/negativ/Developer/ES/es-current/config/scripts/tag_sort.groovy]

Now we are ready to use this script in our queries. You may remember that we used exactly the same script in the Sorting data section in Chapter 4, Extending Your Querying Knowledge. Now the modified query that uses our script stored in the file looks as follows:

curl -XGET 'localhost:9200/library/_search?pretty' -d '{
  "query" : {
    "match_all" : { }
  },
  "sort" : {
    "_script" : {
      "script" : {
        "file" : "tag_sort"
       },
       "type" : "string",
       "order" : "asc"
     }
  }
}'

We will return to this, but first, the next possible way of defining inline scripts.

Inline scripts are a more convenient way of using scripts, especially for constantly changing queries and for ad-hoc queries. The main drawback of such an approach is security. If we allow users to run any kind of query, including scripts, we can expose our Elasticsearch instance to attackers. Such attacks can execute arbitrary code on the server running Elasticsearch with rights equal to the ones given to the user running Elasticsearch. In the worst case scenario, the attacker could use security holes to gain super user rights. This is the reason why inline scripts are disabled by default. After careful consideration, you can enable them by adding:

script.inline: on

Add the preceding command line to the elasticsearch.yml file.

After allowing the inline script to be executed, we can run a query that looks as follows:

curl -XGET 'localhost:9200/library/_search?pretty' -d '{
  "query" : {
    "match_all" : { }
  },
  "sort" : {
    "_script" : {
      "script" : {
        "inline" : "_doc.tags.values.size() > 0 ? _doc.tags.values[0] : "u19999""
       },
       "type" : "string",
       "order" : "asc"
     }
  }
}'

The last option for defining scripts is storing them in the dedicated Elasticsearch index. For the same security reasons, dynamic execution of the indexed scripts is by default disabled. To enable the indexed scripts, we have to add a similar configuration option to the one we added to be able to use the inline scripts. We need to add the following line to the elasticsearch.yml file:

script.indexed: on

After adding the preceding property to all the nodes and restarting the cluster, we will be ready to start using the indexed scripts. Elasticsearch provides an additional, dedicated endpoint for this purpose. Let's store our script:

curl -XPOST 'localhost:9200/_scripts/groovy/tag_sort' -d '{
  "script" :  "_doc.tags.values.size() > 0 ? _doc.tags.values[0] : "u19999""
}'

The script is ready, but let's discuss what we just did. We sent an HTTP POST request to the special _scripts REST end-point. We also specified the language of the script (groovy in our case) and the name of the script (tag_sort). The body of the request is the script itself.

We can now move on to the query, which looks as follows:

curl -XGET 'localhost:9200/library/_search?pretty' -d '{
  "query" : {
    "match_all" : { }
  },
  "sort" : {
    "_script" : {
      "script" : {
        "id" : "tag_sort"
       },
       "type" : "string",
       "order" : "asc"
     }
  }
}'

As we see, the query is practically identical to the query used with the script defined in a file. The only difference is that we provided the identifier of the script using the id parameter instead of providing the file name.

We need to implement at least two classes to create a new native script. The first one is a factory for our script. For now, let's focus on it. The following sample code illustrates the factory for our script:

package pl.solr.elasticsearch.examples.scripts;

import java.util.Map;

import org.elasticsearch.common.Nullable;
import org.elasticsearch.script.ExecutableScript;
import org.elasticsearch.script.NativeScriptFactory;

public class HashCodeSortNativeScriptFactory implements NativeScriptFactory {

    @Override
    public ExecutableScript newScript(@Nullable Map<String, Object> params) {
        return new HashCodeSortScript(params);
    }

  @Override
  public boolean needsScores() {
    return false;
  }

}

The essential parts are highlighted in the code snippet. This class should implement the org.elasticsearch.script.NativeScriptFactory class. The interface forces us to implement two methods. The newScript() method takes the parameters defined in the API call and returns an instance of our script. Finally, needsScores() informs Elasticsearch if we want to use scoring and whether it should be calculated.

Now let's look at the implementation of our script. The idea is simple – our script will be used for sorting. Documents will be ordered by the hashCode() value of the chosen field. The documents without a value in the defined field will be first on the results list. We know the logic doesn't make too much sense, but it is good for presentation as it is simple. The source code for our native script looks as follows:

package pl.solr.elasticsearch.examples.scripts;

import java.util.Map;

import org.elasticsearch.script.AbstractSearchScript;

public class HashCodeSortScript extends AbstractSearchScript {
  private String field = "name";

  public HashCodeSortScript(Map<String, Object> params) {
    if (params != null && params.containsKey("field")) {
      this.field = params.get("field").toString();
    }
  }

  @Override
  public Object run() {
    Object value = source().get(field);
    if (value != null) {
      return value.hashCode();
    }
    return 0;
  }

}

First of all, our class inherits from the org.elasticsearch.script.AbstractSearchScript class and implements the run() method. This is where we get the appropriate values from the current document, process it according to our strange logic, and return the result. You may notice the source() call. It is exactly the same _source parameter that we used when dealing with non-native scripts. The doc() and fields() methods are also available and they follow the same logic we described earlier.

The thing worth looking at is how we've used the parameters. We assume that a user can put the field parameter, telling us which document field will be used for manipulation. We also provide a default value for this parameter.

We said that we will install our script as a part of a plugin. This is why we need additional files. The first file is the plugin initialization class where we tell Elasticsearch about our new script:

package pl.solr.elasticsearch.examples.scripts;

import org.elasticsearch.plugins.Plugin;
import org.elasticsearch.script.ScriptModule;

public class ScriptPlugin extends Plugin {

  @Override
  public String description() {
    return "The example of native sort script";
  }

  @Override
  public String name() {
    return "naive-sort-plugin";
  }

  public void onModule(final ScriptModule module) {
    module.registerScript("native_sort", HashCodeSortNativeScriptFactory.class);

  }

}

The implementation is easy. The description() and name() methods are only for information, so let's focus on the onModule() method. In our case, we need access to the script module – Elasticsearch service with scripts and scripting languages. This is why we define onModule() with one ScriptModule argument. Thanks to Elasticsearch magic, we can use this module and register our script so it can be found by the engine. We have used the registerScript() method, which takes the script name and the previously defined factory class.

The second needed file is a plugin descriptor file: plugin-descriptor.properties. It defines the constants used by the Elasticsearch plugin subsystem. Without more thinking, let's look at the contents of this file:

jvm=true
classname=pl.solr.elasticsearch.examples.scripts.ScriptPlugin
elasticsearch.version=2.2.0
version=0.0.1-SNAPSHOT
name=native_script
description=Example Native Scripts
java.version=1.7

The appropriate lines have the following meaning:

And that's it. We have all the files needed to run our script. Please note that you can have more than a single script packed as a single plugin.

Now it's time to install our native script embedded in the plugin. After packing the compiled classes as a JAR archive, we should put it in the Elasticsearch plugins/native-script directory. The native-script part is a root directory for our plugin and you may name it as you wish. In this directory you also need the prepared plugin-descriptor.properties file. This makes our plugin visible to Elasicsearch.

After restarting Elasticsearch (or the whole cluster if you run more than a single node), we can start sending the queries that use our native script. For example, we will send a query that uses our previously indexed data from the library index. This example query looks as follows:

curl -XGET 'localhost:9200/library/_search?pretty' -d '{
  "query" : {
    "match_all" : { }
  },
  "sort" : {
    "_script" : {
      "script" : {
        "script" : "native_sort",
        "lang" : "native",
        "params" : {
          "field" : "otitle"
        }
      },
      "type" : "string",
      "order" : "asc"
    }
  }
}'

Note the params part of the query. In this call, we want to sort on the otitle field. We provide the script name native_sort and the script language native. This is required. If everything goes well, we should see our results sorted by our custom sort logic. If we look at the response from Elasticsearch, we will see that the documents without the otitle field are at the first few positions of the results list and their sort value is 0.