Accelerating Communication-Intensive Applications via Novel Data Compression Techniques

Processor speed has traditionally grown at a rate faster than that ofcommunication speed in computer and supercomputer networks, and it isexpected that this trend will continue even stronger, as we move into theexascale age in the upcoming decade. This has resulted in what is known asthe "communication gap" for communication-bound HPC applications: theircommunication-to-computation time-ratio is so large, that the processorsremain mostly sub-utilized, with lots of "disposable" FLOPS available. Inthe last few years, scientists have proposed to use these disposable FLOPS(which otherwise would be wasted idling) to compress and decompress thecommunicated data so to effectively speed up the underlying application.Although the idea bears tremendous potential, efforts in this direction haveconsistently rendered very poor results, with typical resulting speedupsaveraging below 1.5x. In this project, we identify the strongest reasons whytraditional data compression has fallen short in terms of speedupperformance for HPC, and propose novel techniques particularly crafted forgroundbreaking performance within the HPC framework. Preliminary resultsshow that these techniques break the 10x speedup markup consistently for awide class of HPC applications of primary importance to NASA. We propose todevelop the theory and methods behind these techniques, which ultimatelywill result into a library product for transparent acceleration of HPCcommunication platforms, such as MPI. Accelogic has already secured PhaseIII private capital in the amount of $1 million for the deployment of suchpotentially revolutionary product, following a successful Phase II.