Deep Agent with Self-learning for Human Events Recognition (DASHER)

Automated analysis of human activities and events on the ground from aerial videos is key to extracting critical situational awareness and gathering actionable intelligence. It poses multiple challenges, including dynamic backgrounds that can be confused with foreground, multiple simultaneous activities in the field of view, occurrence of unknown activities, and large data volume and velocity. The state-of-the-art tools are unreliable, limited in scope, relies on extensive training data, and primarily designed for videos taken at the ground level or for close-up shots. Our proposed DASHER technology will address all of the above challenges in activity recognition for aerial videos via an automated system of smart agents that are capable not only of modeling and inferring activities at various spatial, temporal, and contextual scales but also discovering and learning new activities with minimal human intervention. DASHER's key components are self-learning smart agents for real-time detection of human activities and events at multiple spatiotemporal and contextual scales; a graph fusion engine for collective analysis and inferring high-level semantics; and a dynamic context model for robust activity and event classification. DASHER requires minimal human intervention and can operate under small amount training data.