Real-time Scene Labeling and Passive Obstacle Avoidance in Infrared Video

IR sensor technologies are increasingly leveraged by the Army, for vehicle navigation in blackout conditions, and intelligence exploitation.In recent years, computer vision technologies have been pursued to automate the extraction of meaningful information from sensor feeds, which can aid human interpretation, reduce workload, improvement effectiveness and reduce fatigue.While much progress has been made in object recognition technologies for static IR images, many technical challenges remain for full semantic segmentation and labeling of IR video in real time to support detection of obstacles and threats.In this effort, we propose to advance the state of the art in the automated processing of IR video, by combining an appropriate segmentation algorithm with a novel Progressively Expanded Neural Network (PEN Net) approach to accelerate recognition processing, improve recognition accuracy, and enable near-real time training updates.The algorithm will output fully labeled scenes and support user interaction features for training updates.We will also define and implement post processing algorithms that operate on the labeled data and contextual information for recognition of objects and threats.Our solution will be vetted with the operational community at the end of Phase I, and then fully-developed in Phase II.