ExORun: A Cross-cutting Software Suite for Exascale Optimizing Runtime

Availability of extreme scale computers in the near future will enable simulations with high societal impact. However, the nature of many applications that will run on these systems, and static and dynamic variability of the hardware, along with the huge power requirements and possibility of component failures, create bottlenecks that limit application scalability and efficiency. ECP Software Technology projects address these problems, but need to be complemented with support for across-node runtime adaptivity to enable effective harnessing of exascale machines for many applications. This project will develop a suite of libraries called ExORun, which will support across-node runtime adaptivity including dynamic load balancing, based on over-decomposition, for applications written using MPI, MPI+X, Charm++, AMPI, as well as other programming models. Task-based dynamic programming models, specifically Charm++, which can address the challenges, will be readied for extreme-scale challenges in a commercially sustainable software, and to use the ExORun libraries. Libraries will be developed for a) job-level optimizations for energy/power/thermal issues, to interoperate with node and machine level capabilities provided by vendors and DoE’s Argo project; and b) supporting resilience to hard and soft faults. The initial effort will focus on developing an initial suite of scalable load balancing strategy libraries. Then to develop MPI interface necessary for supporting over-decomposition, instrumentation support, and generic location management, thus providing MPI+X applications with dynamic load balancing capabilities. A prototype task library will be developed that uses control over within-node resources to optimize across-node metrics for multicores and accelerators. Design of libraries for resilience and power management will be completed and prototypes evaluated. Exascale software is a nascent niche ecosystem. By being one of the few industrial players within that ecosystem, we expect to see benefits in the rate of adoption of our technology. Runtime adaptivity will allow a larger class of applications to scale to exascale than otherwise feasible, which will lead to both profitability for Charmworks, but also societal benefits because of novel science/engineering enabled.