STAR: Strategies and Toolkit for Adversarial Robustness

Aptima, Inc. and PatchPlus Consulting plan to address the Navy’s problem of defending current and future autonomous aerial systems against counter-counter measures by building a large language model (LLM)-driven development agent named STAR (System for Threat Analysis and Response) that will function to not only generate a taxonomy of the attack surface, but also form a full, future-adapting test suite for AI developers through specialized agents. The first agent, the Security Knowledge Generator (SKG), will construct and continually update a structured, annotated knowledge representation of the attacks, defenses, example vignettes, systems, and functions for Naval IAS. The second agent, the Scenario and Code Generator (SCG), will create the security scenarios for a given autonomy system. The scenarios will include the step-by-step instructions for attacking or defending the autonomous system and AI-based functional modules of interest that will include text descriptions for high-level system components and the code templates for the steps that correspond to individual AI modules. Finally, the third agent, the Evaluation Report Generator (ERG), will parse the results of the experiments and user-system analysis to construct the aggregated, user-readable evaluation report cards describing the IAS assessment process and summary of results.