Recognition of Human Gestures for Device Control, Interacting with Virtual Worlds, and Interpreting Human Activities

We propose the development of a system for visually tracking and interpreting human motion for the purpose of controlling devices, interacting with virtual worlds, and interpreting overhead reconnaissance video data. That is, we intend to implement a gesture recognition and device control system. In many application, using gestures for device control is more practical than other forms, such as speech or keyboard, because of a noisy background environment and because many commands are more intuitively represented as gestures. For flexibility and ease of use, the recognition system will not require an enhanced environment of attached data-gloves or beacons. Instead, we will implement a gesture recognition system which performs in a cluttered background environment using standard video and an expensive PC based system. We intend to use the dynamic properties of gestures for real-time recognition, transforming dynamic parameters into reference trajectories used for device control and interaction in virtual worlds. Further, we propose the development of a system which uses the dynamic properties of combinations of gestures for differentiating between humans, animals, and mobile mechanical objects. Such a system would be used to quickly categorize and prune the search space of objects requiring observation during overhead reconnaissance.