Scientific Data Server (CHIMERA)

The Air Force Test Center (AFTC) is engaged in the acquisition and analysis of aeronautical telemetry data from flight tests – specifically, HDF5 and IRIG data. While there is some advanced data management tools for HDF5 data, IRIG data, on the other hand, currently lacks a system to streamline its ingestion, manipulation, and analysis. Additionally, no centralized data management construct exists across the two data types, nor is there a mechanism to interrelate the data. We propose CHIMERA as a high-performance distributed data server solution that addresses multiple problems intrinsic to large datasets. In the Phase I effort, the team performed spike tests to prove out an approach to address data format complexity up front and allow the system to leverage mainstream proven Big Data technologies for scalability and performance.  The approach then builds out further by overlaying advanced data analytics techniques to provide users with far more control and insight into the underlying data.  CHIMERA segregates data discovery from data querying, data analysis, and data management while providing a unified interface for all three.