Multi-Modal Immersive Intelligent Interface for Remote Operation

This research will develop a user interface, the Multi-Modal Immersive Intelligent Interface for Remote Operation (MIIIRO), for planning and controlling unmanned aerial vehicles (UAVs), unmanned tactical aircrafts (UTAs) and other remote systems. MIIIRO consists of (1) a community of intelligent agents which are aimed at reducing work and information overload, (2) an immersive environment which induces a sense of presence in the engagement area, and (3) multi-modal inputs, including head tracker and joystick, which enable efficient interactions. An innovative open multi-agent architecture is proposed to facilitate communication and coordinate activities among the intelligent . The control strategy for the UAVs, UTAs, and other remote systems will be based on the task lines and motion guides technology developed by NASA. MIIIRO uses a consumer-grade helmet-mounted display to provide the immersive environment and multi-modal inputs. The intelligent agents will be implemented in Java while the virtual worlds in VRML. Because MIIRO uses commercial-off-the-shelf products and standardized platform-independent technologies, compatibility and interoperability will be ensured.