A Statistical Grammar Approach for Human Activity Discovery from Video

IAI and its academic partners propose to develop a human activity discovery system based on linguistic human activity language and statistical grammar model. This system will estimate human body pose and anthropometry from video data and infer human behavior using a statistical grammar based approach. This approach fuses two recent advances in computer vision research. First, our advanced tracking algorithms will present a series of hypotheses of human poses recognized within a video sequence; these detection and tracking algorithm will use modern segmentation techniques to extract labeled silhouettes, feeding a 3D reconstruction algorithm that relied on local optimal search of the pose space. Second, algorithms that operate on our statistical model of human motion and behavior, the Human Activity Language, combine these pose estimates into unified, robust estimate of pose and activity. We hypothesize that combining action and pose estimation under a single framework will more robustly identify both. Our algorithms will be useful in both recognition and in reproduction of human action. Upon success, our research will be transitioned into fields of persistent surveillance, health care, robot intelligent interactions with human and modeling and simulation.