At the time of writing, Spark's current version is 2.1. Please check the latest version from Spark's download page at http://spark.apache.org/downloads.html. Binaries are developed with the most recent and stable version of Hadoop. To use a specific version of Hadoop, the recommended approach is that you build it from sources, which we will cover in the next recipe.
All the recipes in this book are developed using Ubuntu Linux, but they should work fine on any POSIX environment. Spark expects Java to be installed and the JAVA_HOME environment variable set.
In Linux/Unix systems, there are certain standards for the location of files and directories, which we are going to follow in this book. The following is a quick cheat sheet:
| Directory | Description |
| /bin | This stores essential command binaries |
| /etc | This is where host-specific system configurations are located |
| /opt | This is where add-on application software packages are located |
| /var | This is where variable data is located |
| /tmp | This stores the temporary files |
| /home | This is where user home directories are located |