Markerless 3D Human Body Modeling for Human Behavior Understanding from Video Imagery

IAI and its collaborator, Dr. Aloimonos at the University of Maryland Computer Vision Lab, propose to develop a markerless 3D human body shape estimation and behavior detection system. This system will estimate human body pose and anthropometry from video data and infer human behavior using a model-based approach. This approach fuses two recent advances in computer vision research. First, our advanced tracking algorithms will present a series of hypotheses of human poses recognized within a video sequence; these detection and tracking algorithm will use modern segmentation techniques to extract labeled silhouettes, feeding a 3D reconstruction algorithm that relied on local optimal search of the pose space. Second, algorithms that operate on our statistical model of human motion and behavior, the Human Activity Language, combine these pose estimates into unified, robust estimate of pose and activity. We hypothesize that combining action and pose estimation under a single framework will more robustly identify both. Our algorithms will be useful in both recognition and in reproduction of human action. We will transition our research into the fields of persistent surveillance, health care, and modeling and simulation. BENEFIT: Reconstructing 3D human body model and motion from video sequences will allow for rapid location of concealed human-borne threats in a crowd from surveillance systems, significantly increasing the effectiveness and efficiency of current surveillance systems, while reducing total cost of ownership. Applications would include military reconnaissance and surveillance, border and coast patrol, law enforcement and homeland security. The Phase I effort will result in a suite of algorithms and a software prototype for surveillance that demonstrate an advancement in the state of the art in human tracking, pose estimation, activity modeling, and pattern recognition techniques. This research will have tremendous potential in many commercial applications such as intelligent video analysis, video browsing and indexing, search and rescue missions, human figure creation/animation for entertainment, and virtual design and fitting for the apparel industry.