Operator Interface for Flexible Control of Automated Sensor Functions

ABSTRACT: In the DoD vision for unmanned systems, there is the demand for a single operator to manage multiple ISR assets, including UAV platforms and their sensor systems. Some research has been conducted in supervisory control and delegation interfaces for unmanned systems, but these have focused on platform mobility rather than sensor systems. Furthermore, these interfaces have focused on high-level"play-calling"but not on all the aspects of interaction and feedback that make delegation successful in human systems. We propose to leverage our existing work in supervisory control to develop the Smart Interaction Device for ISR (SID-ISR): a natural, intuitive user interface for flexible delegation of multiple UAVs for ISR missions. SID-ISR will enable a user to dialogue with the ISR systems to delegate tasks at different levels to support adaptable autonomy, using multiple intuitive modes of interaction such as sketch and speech. SID-ISR will also provide feedback to the user on demand or per protocol to give transparency into the workings of the autonomy and its progress on tasks. In Phase I, we will design and evaluate a prototype to asses the utility and feasibility of our proposed approach. BENEFIT: A key obstacle to the goal of a single operator managing multiple aircraft and sensor systems is the manpower intensity of current operations. Current battlefield UAVs require multiple people to operate, including navigation, system health monitoring, and payload and mission management. An intuitive, flexible multi-modal delegation system for supervisory control of multiple aircraft and sensor systems has the potential to revolutionize how UAVs are current employed. Changing the way in which an operator can interact with these systems and subsystems, frees the operator to work at the level of supervisory control, thereby increasing the user"s span of control. This will allow a UAV operator to work with multiple UAVS much like a human team lead can manage multiple subordinates. SID-ISR, fielded as part of a multi-UAV control station, would help make unmanned systems easier to use. SID-ISR could also be readily extended to other ISR applications, in other domains such as ground or space-based operations. As commercial markets grow in the coming years, SID-ISR would be valuable in commercial surveillance, agricultural crop management, and border patrol where the sheer size of the space to monitor requires multiple sensor assets.