AI-Fire: Advanced Fire Ignition Video Analysis through Multimodal Spatiotemporal Detection & Localization Using Transformer-Enhanced YOLO

Analyzing fire ignition events in aircraft dry bays due to impacting ballistic threats presents significant challenges, particularly in capturing the spatiotemporal characteristics and dynamics of fragment flashes and armor-piercing incendiary (API) projectile function clouds. Current approaches are limited to characterizing these events accurately under varying light conditions due to numerous variables and evolving dynamics, and they fail to provide precise spatial and temporal analyses. InfoBeyond introduces AI-Fire to address these challenges using a new Transformer-enhanced YOLO deep learning architecture for multimodal spatiotemporal detection and localization. Particularly, AI-Fire employs a Multimodal Spatiotemporal Transformer Feature Extractor with 3D Shifted-Windows (Swin) Transformer frameworks to extract features from input video sequences, enabling enhanced small object feature extraction and comprehensive spatiotemporal analysis from high-speed visual and infrared camera data. Unlike traditional methods limited to simple feature fusion, AI-Fire integrates features from multiple modalities using Modality-Channel-Spatial Attention. Further, our AI-Fire leverages a 3D YOLO Detection Framework with orthographic features, 3D spatial-temporal attention, and Non-Maximum Suppression (NMS), including the temporal dimension, which traditional single-frame methods lack. This innovative approach improves the characterization of ballistic fragment flash and API projectile functions, accurately tracking fuel leakage during fire tests. Importantly, AI-FireÆs detailed, parameterized 3D detection results including the size, location, volume, and intensity, provide deep analytical insights, such as identifying cause-consequence relationships and event dependencies often obscured in high-speed ignition processes. This enhances the understanding of fire ignition dynamics, aiding in developing predictive models. This leads to better safety protocols, optimized fire suppression strategies, and advanced material designs. The US Air Force can use these insights to enhance aircraft survivability in combat scenarios, ensuring better preparedness and response strategies.