In step 1, you need to specify the main class that should run while executing the JAR file. In the preceding demonstration, SparkExample is our main class that invokes a training session. You may come across exceptions that look as follows:

Exception in thread “main” java.lang.SecurityException: Invalid signature file digest for Manifest main attributes.

Some of the dependencies that were added to the Maven configuration may have a signed JAR, which may cause issues like these.

In step 2, we added the filters to prevent the addition of signed .jars during the Maven build.

In step 3, we generated an executable .jar file with all the required dependencies. We can submit this .jar file to spark-submit to train our networks on Spark. The .jar file is created in the target directory of the project:

The Maven shade plugin is not the only way to build an uber-JAR file. However, the Maven shade plugin is recommended over other alternatives. Other alternatives may not be able to include the required files from source .jars. Some of those files act as dependencies for the Java service loader's functionality. ND4J makes use of Java's service loader functionality. Therefore, other alternative plugins can cause issues.