DARPA-USAF Integration: Collaboration and Secure Tasking for Multi-Agent Swarms

The proposed STTR Phase II submitted by Visual Semantics, Inc. (dba Third Insight) and its non-profit partner, the Institute for Human and Machine Cognition (IHMC) based in Pensacola, FL, seeks to integrate Explainable AI (XAI) and human-machine teaming with multi-agent, swarm-based command and control (C2) developed under ongoing DARPA and USAF SBIR Phase II projects.   Third Insight’s GenesisTM autonomy software is a hardware agnostic “software plugin” that provides COTS and BlueUAS unmanned aerial and ground vehicles (UXVs) with 2D/3D perception, intelligent planning, and reasoning capabilities. Genesis is a patented neuro-symbolic reasoning engine, which fuses pre-existing knowledge (semantic graphs) with robust perception (deep learning) to give UXVs real-time, visual scene understanding and decision-making capabilities. Our partners at IHMC are performers on two DARPA projects (CREATE and ASIST) that produced highly scalable and robust frameworks for teaming and collaborative tasking of autonomous UAVs.  Their well-known Capture-the-Flag games (both real and simulated) were the focus of our Phase I effort. Phase II will include additional teaming simulations and Search & Rescue (SAR)-focused field tests. Third Insight and IHMC engaged technical SMEs from AFRL Sensors Directorate during Phase I, and subsequently presented Phase I results (and Phase II plans) to AFSOC and SOCOM customers.  Major Devin Beckwith, A5/AISUM Lead and our AFRL TPOC have each signed the attached MOU.  AFRL will additionally budget FY22 contract funds in support of Phase II SAR field tests and demonstrations.  The Defense Innovation Unit (DIU) and two state entities (TX-SAR and MD-SAR) also support the Phase II impact on both offensive and defensive swarm applications. USAF Lt Col Alex “Stoiky” Goldberg, Joint Technology Acquisition Innovation Officer with DIU, is the TPOC for two of Third Insight’s SBIR Phase II’s, and has provided a letter of support to highlight this project’s urgency. SOCOM has a national defense-related mission in the area of Artificial Intelligence for Small Unit Maneuvers (AISUM).  We believe technology development under the subject STTR topic may eventually contribute to solving a mission need. The main goals of our involvement in this project are to: (1) develop multi-agent (swarm) techniques for collaborative tasking and command and control (C2); (2) deploy Explainable AI (XAI) and C2 software components that interoperate with open source architectures to enable decentralized and federated decision-making on board autonomous vehicles; and (3) support AFRL-based testing and hand-off of our technology to AISUM-related Programs of Record (e.g., EOTECH and SOF Warrior). Together, these capabilities and efforts will enable  inter-agent collaboration and self-tasking among platforms whose different capabilities (SWAP, sensing, perception, and reasoning) require real-time coordination and teamwork to address the mission needs.