New HPC Technologies and Trends
In collaboration with the Hewlett Packard Enterprise Data Science Institute, Red Hat presented a webinar regarding the new technologies and trends for HPC. Yan Fisher, a global evangelist at Red Hat, discussed emerging HPC trends, HPC workload container technologies and deployments.
Fisher explained his observations of current trends based on research statistics documented by Hyperion and 451. A prominent trend is the migration of a certain subset of workloads into the cloud, an abstracted set of resources. HPC is shifting its focus to support high workloads and infrastructures, which has led to the increasing demand of container technologies.
Containers are multi-purpose and have capabilities to fundamentally shift HPC workloads. It has distinct advantages in infrastructures and applications. For infrastructures, containers can be portable across different environments. For applications, containers can be accessible and shared easily.
There are multi-phase approaches to enable containers in HPC. The crawl phase enables single host environments and uses Universal Base Images (UBI) to develop containers. The walk phase scales HPC workflows to run on containers and provides the ability to run MPI jobs. The run phase arranges HPC jobs with Kubernetes and provides tight integration with MPI schedulers.
The future of HPC will be driven by AI and machine-learning, cloud utilization and the increased use of enterprise technologies.