Innovative Hands-Free Point-and-Click Computer Control Device Within a Moving Vehicle

A concept for a Gaze-based Point-And-Click (G-PAC) interface component is proposed as an ideal human-computer interaction technology for operations within moving vehicles. In recent years there has been considerable interest in combat vehicle crewreduction through the incorporation of increased automation. As currently envisioned, these automation systems require the use of mouse or trackball. The need to simultaneously manipulate other manual control devices makes the use of these standardmanual computer interface mechanisms impracticable. Beginning with an analysis of hands-free HCI component requirements, in both real-world vehicle contexts and software reference architecture contexts, we propose to develop a multimodal interface basedon eye-tracking and speech recognition technologies. G-PAC will allow crewmembers to perform point-and-click operations on graphical interface objects in a manner similar to mouse-based systems, while simultaneously performing manual control operations.G-PAC builds on earlier research performed by the proposed research team which demonstrated the feasibility of building versatile control mechanisms by combining eye-control and voice-control techniques. The G-PAC interface addresses the extension ofprevious eye-voice interface research into the context of combat vehicle systems and plug-and-play architectures. The proposed Phase I effort will culminate in a proof-of-concept demonstration that will show multimodal interaction in a simulated vehiclecontext.Where human operators are required to continuously manipulate manual controls while interacting with computer-based systems, the need for hands-free interface technology is clear. G-PAC is an ideal interface technology in these situations. Inaddition to a wide range of military contexts, potential applications are envisioned for, among others, firefighting and police operations domains. G-PAC also offers the opportunity of opening the computer market up to a large disabled population whichcannot function with mouse-based systems.