Recognition of Computer-Based Human Gestures for Device Control and Interacting with Virtual Worlds

We propose the development of a system for visually tracking and interpreting human gestures for the purpose of controlling devices, improving training methods, and interacting with virtual worlds. That is, we intend to implement a gesture recognition and device control system. In many applications, using gestures for device control has many practical advantages over other -forms of input, such as speech or keyboard. These advantages include longer input device life due to the lack of moving parts, improved hygiene in systems used by the public, less system degradation by a noisy background environment, and more intuitive commands represented as gestures. We will implement a gesture recognition system which performs in a cluttered background environment using standard video and an inexpensive PC-based system. We intend to use the static and dynamic properties of gestures for real-time recognition, transforming dynamic parameters into reference trajectories used for device control and interaction in virtual worlds. Further, we propose the development of a system which uses both static and dynamic gestures to control the information provided on a public information kiosk. For example, detailed maps could be displayed and manipulated with gestures. BENEFITS: The fundamental technology developed under this Phase I is important because it facilitates a more dynamic interaction between humans and devices. As a result, it would allow many commercial areas to profit, such as entertainment (video games and virtual reality simulators), manufacturing, and telerobotics.