Unified, Cross-Platform, Open-Source Library Package for High-Performance Computing

Changing technologies in the high-performance computing (HPC) space lead to the creation of new tools and libraries to support each emerging paradigm. Many of these are designed only for specific configurations or for use in certain cases, which makes it difficult for a programmer to know which tool or library to use when and how to transition software built for one system to a new configuration. The last several years has seen the emergence of a new trend in HPC design the rise in the use of special purpose add-on hardware optimized for efficient math computations. The most common embodiment is the graphics processing unit (GPU), which is capable of thousands of simultaneous operations. Machines that include such hardware are known as hybrid, which reflects that they are a hybridization of a multi-core CPU in conjunction with a specialized processing device. Effectively utilizing all of these entities simultaneously is challenging as each involves parallel computing, and moreover each requires a different style of parallel computing. To complicate things further, each component may be supplied by a different manufacturer, supported by different software, and the strategy for efficient use can be problem dependent. Despite these complications, this is the current direction of computing. Systems of this nature already represent three of the ten fastest supercomputers in the world and this trend is unlikely to slow down in the near future. Despite this move to hybrid architectures and the diversity of hardware options available, programmer effort for common tasks is often reduced via the use of software libraries. Examples of commonly used libraries are FFT (Fast Fourier Transform), BLAS (Basic Linear Algebra Subsystem), LAPACK (Linear Algebra Package), and IPP/NPP/OpenCV (for image processing tasks). The programmer expects that optimized and functionally correct versions of these and other routines are available for any given hardware. Ideally, the use of libraries lessens debugging time, eases porting to alternative or future hardware, and gives excellent performance because it is tuned by experts in the hardware and the algorithms. In reality, the space is fragmented, which pushes significant burden to the end programmer. For this project, EM Photonics proposed the development of an open source, unified set of fundamental libraries for use on hybrid HPC systems. EM Photonics will provide baseline functionality for common library routines written in OpenCL that will be cross platform and open source. We will also provide a framework that will allow others (researchers, hardware vendors, commercial library providers, etc.) to plug in alternate implementations. In this way, we will ensure that all functionality will be available and share a consistent feel to the developer across different hardware devices and library families while at the same time allowing functionality to be extended and performance enhanced by specialized implementations. This work will be done in conjunction with our partners in the HPC groups at NVIDIA, AMD, and Microsoft with the intention of adding more partners as the project progresses.