Cluster-Scale Acceleration in HPC with Cerebras Systems
The Cerebras Wafer-Scale Engine (WSE) is the largest computer chip in the world. The WSE provides a unique architecture as 850,000 cores are tied together with a high-bandwidth and low-latency network on a single wafer that allows for incredible cluster-scale acceleration. This acceleration on the chip powers the Cerebras CS-2 system, a full AI and HPC solution in a single processor, which is a step forward for engineers in executing fast workflows and creating innovative computations.
In a seminar to the HPC Society of Professionals, Adam Lively, Ph.D., lead of the HPC Solutions Engineering team at Cerebras Systems, showcased the CS-2 system, its solutions addressing constraints of HPC hardware and the Cerebras Software Development Kit (SDK) used to program applications for the wafer.
Appearing as a logical 2D array of individually programmable processing elements, Lively went into depth on the data-flow architecture of the CS-2, which allows for more flexible computing, communication and memory. Furthermore, the accelerator is currently capable of 1.2 Tb/s bandwidth and contains 40 GB of SRAM uniformly distributed across the wafer, delivering performance in a timely manner. With this complex structure, the system can improve poor parallel efficiency and allow applications to go beyond the constraints of data access.
The CS-2 pushes the limits of innovation, having been utilized in advancing AI and HPC research. Lively discusses how the technology was evaluated in multi-energy research with TotalEnergies Research & Technology USA, accelerating CFD models with the National Energy Technology Laboratory (NETL) and upgrading HPC workflows at the Lawrence Livermore National Laboratory (LLNL).
To write custom programs for the wafer, developers can request access to the Cerebras Software Development Kit (SDK), a general-purpose parallel-computing platform and API. The kit consists of the Cerebras Software Language (CSL), libraries and development tools needed to harness the performance of the system’s cluster-scale computing. Currently in version 0.5.1, the Cerebras engineering team strives to improve the package in several areas, such as additional library support and stability, and have plans to release the SDK for public documentation and access.