Onboard Real-time Saliency Architecture Leveraging Track Based Classification

ABSTRACT: In recent years, advances in EO/IR sensor technology have produced systems capable of collecting massive amounts of data that have outpaced embedded processing capabilities. The mismatch between collection and processing capability results in processing only subsets of data collected and/or large processing latency by only obtaining the data after missions are flown. Most of the data collected lies outside the scope of the mission; however the sheer volume of pedestrian data is prohibitive to timely processing of mission critical information present within the data. Advancements in embedded processing technology are needed for future generations of such systems to maximize their ability to disseminate mission critical information during missions. Toyon Research Corporation proposes an embedded architecture that incorporates image processing algorithms with a heterogeneous processing architecture to process raw sensor data into sub regions of mission critical importance with massive throughput. Image processing algorithms, such as registration, background modeling, tracking, and classification; written to leverage the size weight and power (SWaP) benefits of a GPU/DSP/FPGA hardware architecture will allow key functionality to produce saliency in the data. This provides onboard capability for discernment between mission critical and mission indifferent regions on the entire data sets onboard. BENEFIT: The successful completion of this research will result in the development of hardware and software providing a potentially revolutionary improvement in information available to the warfighter from Wide Area Airborne Surveillance (WAAS) data. This capability will be of great interest for future military systems that rely on the automated processing of aerial surveillance video. For example, the developed technology is expected to benefit the Night Stare, Gorgon Stare, Angel Fire, Constant Hawk, TAILWIND, and ARGUS-IS programs as well as numerous smaller platforms as FPA technology continues to advance. Other potential government applications include any situations where surveillance is required by an imaging sensor on an aerial platform, such as border patrol, crime fighting, traffic analysis, and counter-terrorism.