A Platform-Independent Framerwork for Efficient Massively Parallel Execution

Next-generation high-performance computers (HPCs) are built as massively parallel systems where the parallelism exists at many levels. These systems are a collection of nodes all working together. Each node generally contains more than one processor and each processor contains multiple cores. Managing and efficiently utilizing the different parallelism in such a system is a complex task. Further complicating this, we have recently seen the emergence of a new class of processing device, namely numerical co-processors such as the modern Graphics Processing Unit (GPU). GPUs sit as peers to multi-core processors within a node but also have their own programming paradigm. To develop applications that leverage future supercomputers will require utilizing the computational power available in all the devices in a system. To ease this process, EM Photonics proposes the development of tools that allow the programmer to decouple the algorithm they are developing from its underlying implementation on a specific hardware platform. This offers several advantages. First, developers can focus on defining their algorithm without being parallel programming or hardware device experts. The developer does not have to focus on things like memory management or data movement. Second, programs can be quickly adapted to new and future HPC systems as they become available because they are not overwhelmed with hardware specific code. Finally, programs will be efficiently executed through a dynamic scheduler that will protect against workload imbalance that can be modified at runtime without prior knowledge. All this will simplify the development of software for future hybrid HPC systems.