Generative Augmentation by Pose and Style Transfer to Amplify Radiant Imagery (GAP-STAR)

State-of-the-art Machine Learning algorithms have demonstrated impressive capabilities to perform detection, classification, style-transfer, and in general computer-vision tasks on visible-wavelength imagery. However, the applicability of such methods to other imaging domains has received much less attention from the Computer Vision community. The lack of high-quality labeled datasets in non-visible domains is continuing to be a significant factor in slowing down commercial and DoD research in applying bleeding-edge Machine Learning methods to non-visible domains. The DoD has a growing need of robust and reliable autonomous systems, in which computer vision tasks will play an increasingly important role across all armed forces and data domains. From autonomous vehicles, such as UAVs, to battle-space-wide situational awareness for command-and-control tasks, there is a critical need to perceive, detect, understand, classify and react to dynamic threats by exploiting a range of sensors and imaging modalities that span a large portion of the electro-magnetic spectrum. In the proposed work Lynntech will evaluate the robustness and limitations of transfer learning, style transfer, and other related methods spanning a number of different tasks of interest to DARPA. Tasks of particular interest include classify and determine the pose of objects as imaged by non-visual wavelengths sensors. This problem becomes particularly difficult for certain classes of objects for which only a few images exist, such as Low-Shot instances. Lynntech will evaluate strategies such as data augmentation, simulations, and style transfer to be able to generate new labeled datasets in the imaging domains of interest, evaluate the performance of existing classifiers, and foster the development of new, more robust, and less brittle algorithms, trainable with limited initial data sets. During the first year of this Phase II effort, Lynntech will partner with other commercial entities to lay down the groundwork in generating labeled data sets in non-visible domains using simulations and style transfer. In year two, Lynntech will focus on the development and strengthening of classifier and detectors in such domains. The objective of the project is to create usable tools that can bridge the information gap between the visible domain and other imaging domains.