HERMIONE: Human-awarE Reasoner for Multi-modal Interactive Operations in Networked Environments

Data collection capabilities for intelligence, surveillance, and reconnaissance continuously improve, generating more data, faster, and across an ever-increasing spectrum of modalities than can be handled by human analysts alone. While personal digital assistant technology has emerged in the commercial domain to support the basic and consistent needs of their users, it is a challenging proposition to design such systems to help human decision-makers and cognitive workers, in data-rich environments that are dynamic, noisy, uncertain and ambiguous, and where the cost of a mistake is measured in human lives. In response to this need, Aptima and its partners, PatchPlus Consulting, ASTi, and Raytheon, propose to develop the Human-awarE Reasoner for Multimodal Interactive Operations in Networked Environments (HERMIONE), a conversational digital assistant that supports analysts by proactively bringing critical information to their attention in a natural manner so as to optimize overall system performance. HERMIONE will quickly provide information on demand and streamline routine tasks, while engaging analysts in multimodal conversations that best fit their current workflow and objectives. Ultimately, HERMIONE will help analysts to understand the complex information environment, and improve efficiency of decision-making and the quality of the reports they produce.