Automated Analysis and Classification of Anomalous 3-D Human Shapes and Hostile Actions

ABSTRACT: We propose a robust solution for estimating 3D shapes of human targets from multiple, synchronized input video streams. The goal of the project is to apply 3D pose and shape of human targets for automated inference of different attributes that would enable detection of anomalous behavior. The 3D representation of human targets is obtained as visual hull extracted from silhouettes of multi-view images. Low level image features are used to initialize 3D pose of the targets. The pose is refined using top-down algorithms. We have created statistical shape models using CAESAR dataset that models shape variations among different demographics. We search over this detailed 3D shape model to estimate optimal matching 3D shape of the subject. The shape model is used to detect anomalous shapes, attributes like gender, age and ethnicity. We train graphical chain models to recognize different actions. We have proposed modular software architecture with user friendly GUI, and open data formats to enhance interoperability. Preliminary results in Phase I validate the feasibility of this approach. Phase II work will include an effort to refine and optimize the framework to achieve near real-time processing and to enhance the system"s robustness to more complex poses and in varied environment. BENEFIT: The proposed work addresses the technical challenges of human pose and shape estimation in video imagery. It will enable us to detect human activities and to identify suspicious and hostile human behaviors for military and law enforcement applications. The technology has wide-ranging application beyond intelligent surveillance system: a) Improved human computer interaction The framework will facilitate development of more accurate vision based systems to recognize different gesture and motion in 3D. This has vast potential use in role-playing games where the movements of the user in the physical domain are appropriately reflected as an action in the virtual environment. b) Clinical diagnosis and movement analysis For identifying the underlying causes for walking abnormalities in clinical patients. The results of gait analysis have been shown to be useful in determining the best course of treatment in these patients. c) Intelligent training systems for sporting activities The analysis of sports-related movements often entails analyzing a variety of highly dynamic movements. Motion analysis provides the tools for the sports medicine and performance professionals to perform accurate functional evaluations/analyses for clinical and research oriented purposes. d) Realistic animation Cost-effective solution to importing realistic body movements in animated characters in videos. Human gait modeling can be used to simulate realistic walking styles e) Robotic locomotion Design of robot appendages and control mechanisms to allow robots to move fluidly and efficiently similar to humans