Shape and Motion Encoded Methods for Automatic/Assisted Recognition of Human Intent in IR Images

SET Associates proposes to design, develop and evaluate an assisted/automated system for recognition of intents of humans in urban environments. We propose a modular architecture that involves solving the following problems: detection of stationary and moving humans in different cluttered backgrounds, state of the human (whether armed or not) and recognition of the particular weapon being carried and the intent of the human(s) carrying the weapon. In addition, the proposed system will determine if the detected humans are carrying harmless objects. The approach taken for detection of stationary and moving humans and the weapons they may carry uses a combination of shape and motion driven matching techniques. Patterns of motion exhibited by detected humans prior to conducting a hostile act are analyzed using deformable shape analysis methods. Finally, hidden Markov models are trained to recognize one of many hostile acts that the detected target may be planning to undertake. Our design methodology allows for operator intervention and on-line training. The system will be evaluated using data collected by SET Associates and also supplied by the sponsor. Issues such as real-time implementation and robustness to variations in modeled intents will be addressed.