If you plan to develop code for Drill, you will need to set up a Java development environment. You will find it convenient to create two kinds of projects. The first is a clone of the Drill source code. Often you will need to reference Drill code to find how to accomplish a task.
The next two chapters describe how to create user-defined functions and format plug-ins. It is often easiest to debug your custom code as part of the Drill build so that you can get rapid edit-compile-debug cycles, create unit tests, and so on. Later, when your code is stable, you have the option to package the code into the second kind of project: an extension project separate from Drill.
Drill makes use of Maven for the build-and-test process. A detailed description of Maven operations is beyond the scope of this book, but we do cover how to install it for Drill development.
To develop Drill extensions, you need JDK version 1.8 (as of Drill 1.13) installed on your computer. For information on how to install this, see Chapter 2. In addition to the JDK, you will also need to install Maven version 3.3.1 or greater. On Linux machines, you can easily install Maven using yum
or apt-get
, and on Macs, you can install it using brew
.
On Windows, installation is a little more complex. You first must download Maven from the Apache Software Foundation page and unzip it. After you’ve unzipped it, move the uncompressed file to C:Program FilesApacheMaven. Next, you need to add two environment variables to your environment: M2_HOME
and MAVEN_HOME
. These variables should be set to the folder in which you unzipped Maven (in our example, C:Program FilesApacheMaven). Finally, you will also need to update the PATH
variable to include the Maven binaries. To do this, add %M2_HOME%in
to the PATH
variable.
On either system, verify that you have successfully installed Maven by entering mvn
-–version
at the command line. You should a result similar to this:
$ mvn --version Apache Maven 3.5.0 (... Maven home: /usr/local/Cellar/maven/3.5.0/libexec Java version: 1.8.0_65, vendor: Oracle Corporation Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/jre Default locale: en_US, platform encoding: UTF-8 OS name: "mac os x", version: "10.12.5", arch: "x86_64", family: "mac"
Developing for Drill is complex, and you will save yourself a lot of headaches by using a Java integrated development environment (IDE). Although IntelliJ is the most popular IDE for use with Drill, many developers successfully use Eclipse.
For this project, you will need a clone of the Apache Drill project and a Java IDE.
Begin by cloning Apache Drill from the Apache GitHub repo into your own private GitHub repository. This step is not strictly necessary (you can clone directly to your laptop), but it is convenient. Plus, you will need your own GitHub repository if you choose to offer your plug-in to Apache Drill in the form of a pull request.1
First, if you have not done so already, fork the Apache Drill source code into your own GitHub repo by clicking the Fork button shown in Figure 10-1.
When you have forked the repository, open a terminal, navigate to where you want to store your files, and clone your fork with the following command, replacing username
with your GitHub user account:
git
clone
https://github.com/
username
/drill
At this point, you should have the source code downloaded on your machine. Navigate to that folder and enter the following Maven command to build Drill:
mvn clean install -DskipTests
After this command is complete, there will be a tarball in the distribution/target folder. Copy that to where you want Drill to reside and unzip it; you now have a working installation of Drill.
The final step is installing the IDE. As indicated earlier, we strongly recommend that you use either IntelliJ or Eclipse to develop Drill. Installing either of these tools is straightforward and simply involves downloading the installer from the tool’s website and running it. After you have done that, you can import the project by opening the pom.xml file in your Drill fork root directory.
If you intend to develop code to share with the Drill community, you should download the Drill IDE formatters, which will format your code according to the Drill community standards:
Eclipse is able to compile Java code, but it is not able to do other tasks in Maven, such as code generation. A good practice after each update from Git is to rebuild Drill using Maven (mvn clean install -DskipTests
) and then refresh all files in Eclipse by selecting Refresh from the context for the drill-root
project in the Package Explorer view.
Also, in Eclipse, it is convenient to group all Drill projects into a Drill working set using the File → New → Java Working Set command.
Finally, we encourage you to review the Drill developer guidelines to become familiar with developer resources.
1 We are both committers to the Drill project, and we strongly encourage you to think about contributing to the project, as well.