Automated Analysis and Classification of Anomalous 3-D Human Shapes and Hostile Actions

This project aims to develop algorithms and software tools for automated inference of human shape and pose, for anomalous shape and hostile actions identification. The problem is challenging owing to large variation in human body shapes, inaccurate 3D data acquisition techniques, non-observability of body parts, self-occlusion and concavities on body surfaces. We propose a robust solution based on fast and efficient data structures that iteratively estimate 3D shape of human targets using multiple input video streams. The 3D representation of human target is obtained as visual hull extracted using space carving. We augment these with medial axis and hierarchical image descriptors as visual cues for robust 3D pose initialization. We have created a statistical shape model using CAESAR dataset that models shape variations among different demographics including gender, age and race. This will be augmented with data of clothing and accessories. The shape model is used to detect anomalous shapes. We adopt learning based approach for action recognition and train temporal graphical models to recognize hostile actions. We will design a modular software architecture with clearly defined software interfaces, user friendly GUI, and open data formats to enhance interoperability and allow for future extensions with new data and actions. BENEFIT: The technology will provide intelligent video analytics to detect suspicious persons and hostile activities for counter-terrorism in military and law enforcement. The technology is also relevant to many areas in the private sectors, including business security, retail, marketing, healthcare, sports, and video games and entertainment. The technology has wide ranging applications: (1) In surveillance: Identify potential suicide bomber in hotels and around commercial landmarks. Identify hostile or abnormal behavior in and around office buildings, embassies, shopping malls, schools, and tourist districts. (2) Improved human computer interaction: The framework will facilitate development of more accurate vision based systems to recognize different gesture and motion in 3D. This has vast potential use in role-playing games where the estimated shape of the user can be used to create avatars and the movements of the user in the physical domain are translated into actions in the virtual environment. (3) Video monitoring in healthcare and fitness center: Detect people who have slipped, fallen, or are in need of other assistance, in health centers or nursing homes. Improved techniques for monitoring body fat and regions of body putting on weight. (4) Intelligent training systems for sporting activities: The analysis of sports-related movements often entails analyzing a variety of highly dynamic movements. Motion analysis provides the tools for the sports medicine and performance professionals to perform accurate functional evaluations/analysis for clinical and research oriented purposes. (5) Provide body shape data for estimating range of sizes and improved design of clothing, footwear and sporting equipment