??GOTCHA: Graph-based Object-oriented Transformers using Contextual Hierarchical Attention

Commercial imaging satellites and overhead aerial platforms capture millions of images every day. Since the manual exploitation of overhead wide-area imagery data is intractable, Automatic Target Recognition (ATR) models are used to detect and classify static and moving targets. However, small targets of interest - such as vehicles, ships, and small man-made structures - have often only few pixels per frame, they cannot be identified with sufficient quality by ATR or even human operators. AFRL is seeking automated analytics that can fuse data from multiple times and modalities to produce more accurate and faster object detection decisions.   Aptima Inc. and Patch Plus Consulting propose to develop Graph-based Object-oriented Transformers using Contextual Heterogeneous Attention (GOTCHA). GOTCHA will produce small object detection decisions by fusing image parts across time, sources, and multiple complementary modalities. GOTCHA interprets the image inputs as the space-time-modality-patch (STMP) graph, in which nodes are pixel patches from the individual frames connected by heterogeneous relations that capture local context and long-range structures and complementarities of feature content across sources, times, and modalities. GOTCHA generates detection decisions using graph transformer with heterogeneous relational attention, which has linear computational complexity and higher expressive power to capture complex feature dependencies.