Vision with a Purpose: Inferring the Function of Objects in Video

ABSTRACT: ISR analysts are faced with severe challenges due to the volume of data, the speed at which it arrives, and the variety in sensor signatures and scene content. FMV and WAMI analysts face a fire hose of pixel data. Kitware has developed significant capabilities for ISR video exploitation, with a focus on understanding the function of objects and regions in the scene. These function-based exploitation tools can provide significant help in the automation of both real-time and forensic ISR video exploitation. We plan to leverage and enhance these capabilities to help further advance the state of the art in exploitation technologies. The development and transition of these technologies to operational use could significantly improve the quality of intelligence derived from video streams as well as reduce the resources, both time and personnel, necessary to extract critical intelligence. In terms of commercial opportunities, there are large system integrators who provide FMV exploitation workstations, including Leidos"AIMES system and General Dynamics MAAS systems to the government. Kitware has already received a Phase III SBIR to start transition of our FMV capabilities into the AIMES system. BENEFIT: The primary benefit of the proposed work is a solution to the ISR analyst data-workload challenge. ISR analysts are faced with severe challenges due to the volume of data, the speed at which it arrives, and the variety in sensor signatures and scene content. There are not enoughWAMI analysts to view and exploit all the pixels currently produced by the sensor, even in forensic exploitation. The proposed technology will benefit military analysts on the ground who are responsible for gathering, analyzing, and acting on intelligence, a critical component of almost every mission, though the use of automatic, function-based object detection. The proposed technology will reduce analyst workload while simultaneously increasing the amount of data that can be analyzed, improving security and intelligence gathering. Additionally, there are commercial applications of the technology that provide public benefit and commercial opportunities. These opportunities include the integration of the technology with unmanned aerial systems (UAS) in the commercial sector to benefit precision agriculture; security and monitoring for public safety; exploration, aid efforts, and disaster recovery; and environmental monitoring.