Accelerating MPI and PGAS Libraries for HPC and Deep Learning Applications on Exascale Systems with Programmable Smart NICs

The extremely high compute and communication capabilities offered by modern CPUs/GPUs and high-performance interconnects have led to the creation of HPC platforms with dense many-core CPUs, multiple GPUs and high-performance interconnects. State-of-the-art implementations of popular parallel programming libraries like MPI and SHMEM must be enhanced for various science domains in traditional HPC and DL e.g. molecular dynamics, lattice QCD, seismology, image classification, and fusion research, computer vision) to take advantage of these emerging platforms. Unfortunately, state-of-the-art production quality implementations of the popular programming models do not have the appropriate support for next-generation programmable Network Interface Controllers NICs) like Mellanox Bluefield to deliver the best performance and scalability for applications on such dense many-core CPU/GPU systems. The proposed SMART-CAM product will build upon existing and recognized capabilities to enhance state-of-the-art production quality implementations of the popular MPI and SHMEM programming models to take advantage of next-generation networking and storage technologies available with Mellanox Bluefield network adapters to deliver the best possible scale-up and scale-out for HPC and DL applications on emerging dense many-core CPU/GPU systems. We will develop the following innovations in SMART-CAM to enable scale-up and scale-out of various driving science domains in as indicated above on emerging dense many-core CPU/GPU platforms: 1) Efficient designs for asynchronous progress; 2) Offloading rendezvous communication; 3) In-network collective communication; 4) Novel datatype processing to improve application performance; 5) Offloaded datatype processing; 6) Optimized RMA operations; 7) Accelerated I/O and checkpoint-restart; 8) Online data compression and coalescing; and 9) Carry out integrated development and evaluation to ensure proper integration of proposed designs with the MPI and PGAS libraries. Tasks 1, 2, 3, and relevant portions of 9 will be carried out as part of Phase-1 activities. The transformative impact of the proposed SMART- CAM product will be achieve scalability and performance out of HPC and DL frameworks/applications to take advantage of emerging dense many-core CPU/GPU platforms with programmable NICs like Bluefield. We expect that the solutions that SMART-CAM will provide can reduce the CPU utilization of popular communication middleware by up to 30%. This can result in significant boost to application level performance by making more CPU time available to it. Further, the acceleration of various communication and I/O operations made possible by newer technologies introduced by the Bluefield series of network adapters like RDMA over NVMEoF can reduce the I/O processing time by a factor of up to 5 for I/O intensive workloads like DL training. Furthermore, the availability of programmable ARM cores on the Bluefield SoC can reduce the processing overhead for activities like datatype processing, data compression, and collective communication by up to a factor or 3 for traditional HPC as well as emerging Deep Learning applications.