ConnextLogger: Intelligent Data Logging for MDA Modeling and Simulation Systems

Modeling and Simulation systems, like OSF, can generate incredible amounts of information very quickly; however, the overhead processing required to handle all of this data brings the simulations to their knees. MDA requires an intelligent data logging solution for their modeling & simulation systems that reduces runtime and resource usage (CPU, memory, network, and storage) and collects relevant data, rather than blindly logging all data using a best-effort approach. ConnextLogger is our intelligent, adaptive data-logging platform that will adjust to match the dynamic situation and the analysts’ needs. Our platform will continually collect and asynchronously buffer high fidelity data in circular in-memory caches. This ensures low latency and high throughput while minimizing the impact to the application. The data will be continuously analyzed in real time to intelligently filter out low-valued data. It will seek out anomalies and other events; and, when detected, it will trigger logging of the high-valued, high-fidelity data into permanent-storage. In parallel, a subset of the low-valued data will be logged into permanent-storage even when there are no events or anomalies. This so-called steady state data will provide the analysts needed indicators about the state of the system/objects while minimizing performance impact and resource utilization. Approved for Public Release | 19-MDA-9932 (21 Feb 19)