Reinforcement Learning with Intelligent Context-based Exploration (RL-ICE)

State of the art object detection in satellite imagery currently requires large quantities of hand-labeled satellite images. But what if there exists only very limited satellite imagery of the object, perhaps a single pass? Current deep learning solutions can not learn effective models with this extremely limited data. If, however, there exists model of the object that can be used to synthesize more images, can we create an effective classifier? To address this fundamental problem of training rare-object detection systems with synthetic imagery, SoarTech proposes Reinforcement-Learning with Intelligent Contextual Exploration (RL-ICE). With Deep Reinforcement Learning (DRL), RL-ICE trains an adversarial agent to identify weaknesses in existing object classification systems and generate new, effective imagery to improve performance. As part of this investigation, SoarTech has identified three new innovations: (1) using DRL to control validated image synthesis software, generating highly effective imagery; (2) a novel action space for DRL enabling RL-ICE to learn sequences of image synthesis operations to achieve the goal; and (3) new exploration and reward methods that build on data analysis methods from active learning and additional contextual knowledge to increase the likelihood of convergence to a stable, successful policy.