Rapid Data Analytics Platform using Machine Learning

NASA#39;s vast collection of simulated, experimental, and observational datasets has paved the way for groundbreaking advancements in addressing real-world physical challenges. In particular, NASA has made significant investments in developing state-of-the-art simulation software for climate modeling, aerodynamics, space vehicle launch environments, etc. These software tools along with large computing resources available to NASA scientists enable highly detailed simulations of physical phenomena. There is a need to address the big data bottleneck so that these data can be efficiently stored, shared, and analyzed while providing accessibility to a range of users, from the scientists with large HPC resources at their disposal to downstream users lacking the said resources. In this Phase II SBIR project, RNET and FAU will develop an advanced data management and analytics platform that is designed specifically to address the big data challenges associated with high-fidelity scientific simulation data hosted by NASA and other agencies. We employ a quot;reduce-then-analyzequot; strategy. Our approach combines modern machine learning and traditional computational mechanics based discretization to provide robust and interpretable compression. The use of a computational mechanics framework makes the compressed data amenable to accelerated data analytics without the need to first decompress. The platform aims to provide best-in-class compression-to-error ratios, exceptionally fast point-wise decompression, and a geometry-preserving data format to significantly reduce computational, storage, and bandwidth costs. Seamlessly assimilated into existing scientific data analytics pipelines (e.g., NetCDF, HDF5, PANGEO) and engineered to operate on a wide range of compute resources, this powerful platform will ensure that scientists across all levels of expertise can harness the transformative capabilities embedded in highly detailed simulation datasets.