Panoptic Deep Adaptation (PANDA) Phase II

The DoD lacks the massive annotated data needed to train Deep Neural Networks (DNN) for IR domain applications for which recognition of militarily significant target types is critical. Pixel-wise annotations are close to impossible in IR. When adapting recognition between visual domains, changes in object types, appearance, and poses are not adequately modeled in the new domain. How to suppress the huge dataset bias, overcome the lack of training data, and make rapid adjustment to new target types and environments is critical for making IR AiTR applications effective in Next Generation Combat Vehicle (NGCV) automation.  DYZNE’s PANoptic Deep Adaptation (PANDA) approach will achieve comprehensive transfer learning across EO, IR, and other domains without the need of supervision in the target domain. Deep models are adapted at all representation levels. No pair-wise correspondences are needed between the source and target domains. Pixel level annotation in IR is avoided via novel adversarial learning with cycle consistent constraints. Mission relevant training data at the target domain are automatically generated/augmented through image-to-image translation followed by a data expansion and contraction process. The development of PANDA is a technology enabler that will support the exploitation of vehicle IR and other sensors towards higher-level cognitive processing.