Explainable Query Refinement for Human Machine Teaming

The Intelligence, Surveillance and Reconnaissance (ISR) analysts have a challenging task to extract useful information from huge volumes of data from various sources like Full Motion Video (FMV), Wide Area Motion Imagery (WAMI), satellite imagery, Synthetic Aperture Radar (SAR), and others. Modern Machine Learning (ML) algorithms based on deep learning have greatly advanced computer vision, speech recognition, and other Artificial Intelligence (AI) areas by achieving state of the art performance in many tasks. There is a great need of Human Machine Teaming (HMT) tools to assist such users by making the exploitation process more efficient through automation, while providing user the insight into the automation to establish trust. To address these challenges Kitware, Inc. has partnered with world-renowned computer vision researchers from Rochester Institute of Technology (RIT). We propose a new concept for the analyst test bed where the goal is to improve HMT through iterative query refinement and algorithm explanation features providing insight into the ML algorithms. We will build upon our experience and existing tools and provide image and video retrieval capability through Interactive Query Refinement (IQR) with new features providing explanations about the ML algorithms output.