Acoustic Training Data Prioritization: Automated Tools for Building Representative and Efficient Datasets

To address Navy’s need for high quality compact datasets for training Artificial Intelligence and Machine Learning (AI/ML) models, ARiA will develop and demonstrate AUtomated TOols for Building Representative and Efficient DaTaseTs (AUTO-BRETT) which provide Test and Evaluation (T&E) engineers with metrics and tools for interrogating existing datasets and new unlabeled data.  AUTO-BRETT builds on and extends tools from ARiA’s Dataset Analysis Monitoring Library (DAML) of the Joint AI Test Infrastructure Capability (JATIC) Program of Record to enable pruning and reduction of existing labeled datasets, selection of new data most valuable for labeling, measurement of dataset sufficiency with respect to models; and metrics for monitoring new data for drift.  Using both model-specific and model-agnostic metrics and both tabular and acoustic data, AUTO-BRETT can be used to characterize datasets for classification, joint detection and classification, and other sequences of conditional detectors/classifiers.  AUTO-BRETT will be integrated initially into the AN/SQQ-89A(V)15 development ecosystem and used to develop and optimize the Advanced Capability Build (ACB) Open Dataset.