Memory management is very crucial for distributed training with large datasets in production. It directly influences the resource consumption and performance of the neural network. Memory management involves configuring off-heap and on-heap memory spaces. DL4J/ND4J-specific memory configuration will be discussed in detail in Chapter 12, Benchmarking and Neural Network Optimization.

In this recipe, we will focus on memory configuration in the context of Spark.