According to Douglas M. Jacobsen and Richard Shane Canon, currently, containers are being overwhelmingly used for the web, enterprise, mobile, and cloud applications. However, there are questions being asked and doubts being raised on whether containers can be a viable runtime for hosting technical and scientific computing applications. Especially, there are many High-Performance Computing (HPC) applications yearning for a perfect deployment and execution environment. The authors of this research paper have realized that Docker containers can be a perfect answer for HPC workloads.
In many cases, users desire to have the ability to easily execute their scientific applications and workflows in the same environment used for development or adopted by their community. Some researchers have tried out the cloud option, but the challenges are many. The users need to solve how they handle workload management, filesystems, and basic provisioning. Containers promise to offer the flexibility of cloud-type systems coupled with the performance of bare-metal systems. Furthermore, containers have the potential to be more easily integrated into traditional HPC environments, which means that users can obtain the benefits of flexibility without the added burden of managing other layers of the system (that is, batch systems, filesystems, and so on).
Minh Thanh Chung and the team have analyzed the performance of VMs and containers for high-performance applications and benchmarked the results that clearly show containers are the next-generation runtime for HPC applications. In short, Docker offers many attractive benefits in an HPC environment. To test these, IBM Platform LSF and Docker have been integrated outside the core of Platform LSF and the integration leverages the rich Platform LSF plugin framework.
We all know that the aspect of compartmentalization is for resource partitioning and provisioning. This means that physical machines are subdivided into multiple logical machines (VMs and containers). Now on the reverse side, such kinds of logical systems carved out of multiple physical machines can be linked together to build a virtual supercomputer to solve certain complicated problems. Hsi-En Yu and Weicheng Huang have described how they built a virtual HPC cluster in the research paper, Building a Virtual HPC Cluster with Auto Scaling by the Docker. They have integrated the autoscaling feature of service discovery with the lightweight virtualization paradigm (Docker) and embarked on the realization of a virtual cluster on top of physical cluster hardware.