Knowledge Discovery using Mobile Devices

A critical need exists to better support expeditionary warfighters in knowledge discovery from imagery and sounds local to their area of operation. Intelligence, Surveillance and Reconnaissance (ISR) knowledge databases contain copious amounts of information on landmarks, terrain, events, activities, and entities that must be efficiently and accurately processed and disseminated to the warfighter. Such information sharing is essential for improving situational awareness and overall operational effectiveness - especially in today's urban combat environments. The confusion of the battlefield combined with physical and psychological stresses and fatigue can make this task very difficult. Pongr, Inc. (Pongr) proposes KAVA (Knowledge discovery using Audio and Video Analysis), a mobile application technology that analyzes audio and video data to discover entities and behaviors in a scene. At the core of the system are state-of-the-art artificial intelligence algorithms for understanding audio and video developed by Pongr researchers. These algorithms are capable of recognizing people, places, objects, anomalous or suspicious activities, and sounds. The algorithms translate images and sounds into knowledge objects that can be searched in a larger knowledge store. Discovered information is displayed using an augmented-reality interface. Finally, the system features hands-free operation via speech and gestural interactions.