Accelerating Communication-Intensive Applications via Novel Data Compression Techniques

The traditional view of arithmetic operations dominating the computational cost of numerical algorithms has changed. As performance of new processors increases, we are moving into a new reality in which data movement is expensive and operations are becoming nearly free. In Phase I we discovered a new theory of data compression with unprecedented capability to reduce data movement in distributed HPC applications. Phase I results include: 1) the demonstration via proof-of-concept prototypes of lossless compressive gains of 4x-20x for NAS Parallel Benchmarks, and 2) the formulation of COPA, a Compression OPportunities Auditing & discovery process that facilitates the analysis of any given large-scale code with the purpose of optimizing it for performance via data movement reduction. Phase II targets the consolidation of the new theory through two major thrusts: 1) the development of software that facilitates integrating high-yield compression into HPC codes, and 2) the further infusion of the technology into NASA applications. Thrust 1 will produce a fully functional prototype of a software suite for the automatic/semi-automatic acceleration of HPC codes via compression. The prototype will incorporate ease-of-infusion features (e.g., MPI-enabled compression and decompression routines), as well as ergonomic features that allow the seamless integration of new compression modules into the suite. Thrust 2 will begin with the application of the Chapter 1 of COPA to five HPC codes of utmost importance to NASA, namely Cart3D, FUN3D, USM3D, Enzo, and WRF. Accelogic will collaborate with the developers of these codes to integrate the compression technology under three different levels of "integrability," namely automatic, semi-automatic, and manual. By the end of Phase II, at least one of these applications will undergo the complete COPA optimization process. Complementary Phase II/III funds for $1+ million have been secured to ensure successful commercialization.