Sea and Observe: Littoral Foundations of Structured Understanding and Iterative Discovery for Image Detection, Identification and Annotation

This proposal addresses the algorithm research and development of structure and iterative discovery for image detection, identification, annotation, and understanding. Our approach is inspired from the ability of humans to encapsulate and develop mental models and the ability of AI to algorithmically learn and contribute to the identification of features, textures, and salient and camouflaged characteristics and objects. We believe that explainability and understandability naturally occur with compartmentalization, reduced-order mental models, and an iterative reasoning process. We identified a short list of concepts, mathematics, and technologies with the feasibility to change the paradigm in state-of-the-art human-AI GEOINT image detection, identification, and annotation. The goal is understanding, the ability to explain why, and a positive change in the trust, mitigation, and training cycles. We propose an AI combining data fusion with (1) the use of geometric, topographic, textual, and material information with context; and (2) the use of concept activation vectors and features with the use of salient and camouflage AI/ML image detection. The objective is creating an explainable, understandable image data product from massive image data sets. We identified a need for image analysis of littoral and maritime data, as well as inland data, products, and the general image tradecraft. One compelling area of focus is the striking increase in automatic and autonomous systems for security, search and rescue, ISR, and active and passive observations, especially in gray zones, remote locations, and extreme environments. In Objective 1, we intend to explore the mathematics and database representations for effective, rapid, and iterative database AI. We have identified three areas of algorithm development: geometric, topographic, and iteration-based AI/ML. The geometric task investigates algorithms for advanced classification and annotation using computational geometry models in a digital database. The topographic task investigates the use of monocular and stereo-pair images for 3D point cloud and semantic mapping. The iteration task investigates cyclical detect-discriminate-refine-classify algorithms to permit human-inspired iterative actions and discovery. In Objective 2, we intend to explore the use of context-based object classification and the use of concept activation vectors as inspired by the broad availability of context representations of most human-generated objects, especially common civilian and military issued items. We will investigate the use of concept activation vector algorithms to translate machine-learned features into human-understandable features. Focus is a key human strategy for processing information. We will investigate multipath algorithms for adding algorithmic focus to traditional neural networks. We will investigate the concepts and algorithms developed in Objectives 1 and 2 for camouflaged object detection.