Real-time Overlay of Map Features onto a Video Feed

This Small Business Innovation Research Phase I project will develop a system to overlay geo-referenced informational data on UAV video in real-time. The proposed system will provide enhanced situational awareness and decision making capabilities to remote users by accurately overlaying the information of their choice. The key innovations in this effort include i) feature matching and filtering techniques to reduce error in sensor data, ii) GPU processing for real-time performance with COTS hardware, iii) an API which facilitates future expansion of the system to arbitrary data types and standards such as KML and Shapefiles, and iv) an intuitive and non-invasive user interface enabling users to select and customize data sets to overlay v) a secondary map display to provide additional context by overlaying the geo-registered video onto a wider map view. The Phase I effort will include: concept development, implementation of proposed enabling technologies, demonstration of proof of concept, and analysis of operational feasibility. BENEFIT: Unmanned Aerial Vehicles (UAVs) are a critical component of Intelligence, Surveillance and Reconnaissance (ISR) capabilities for all branches of the armed forces, but the limited viewing angle and resolution of typical UAV video can limit the ability of users on the ground to act. The proposed system would increase the situational awareness of users by providing important contextual and targeted information, overlaid onto the video in a clear and accurate manner, in real-time.The enabling technologies proposed have the following benefits, within this project and in future efforts: Sensor data error mitigation: The project will develop a technique to filter sensor position and orient metadata and match the video to reference imagery to reduce error, which will improve the accuracy of the generated overlay as well as provide more accurate data about the sensor and platform, which has many applications.Real-time GPU processing with COTS hardware: The project will use common GPU hardware to improve overlay quality and performance while freeing CPU resources and keeping size and cost low.Software Development Kit (SDK) for Data Source Ingestion: The system will use a plug-in architecture to enable importing data sources without modifying the system. An easy-to-use SDK will aid the process of creating new plug-ins, and allow users to import arbitrary sources in the future.Intuitive User Interface: The system will incorporate interface design elements from several well-known applications such as Google Earth and Maps, NASA WorldWind, and TiVo with the goal of reducing the end-user’s learning curve by providing them with controls with which they are already familiar. Additionally, user interaction with overlay elements will be achieved through the use of tool tips and text balloons to display additional textual information that would otherwise clutter the video and be difficult to read on moving imagery.Secondary Map Display: An optional map interface will be available to allow users to see additional contextual information, choose information for overlay, and view the geo-registered UAV video in a wider context on the map. Integrated Video and Data Management System (VDMS): ObjectVideo’s VDMS, created to meet the needs of the ARGUS-IS project, will be leveraged for this effort. The system includes integrated management and storage of metadata along with video, as well as a rich player interface with support for metadata and TiVo-style controls for review and measurement tools for geo-registered video. The VDMS will be expanded to support the proposed overlay information, including controlled airspace symbology, restricted airspace boundaries, FAA centers, air corridors with altitude limits, and SAR and IR data.