Information Fusion to Enable Shared Perception between Humans and Machines via Fused Realities Adaptive Modules (FRAM)

ABSTRACT: Fused Informational Realities (FIR) module provides new levels and types of Information Fusion to Enable Shared Perception between Humans and Machines. Innovatively FIR exploits our semi-transparent superimposition fusion methods to enhance human Intel analysts perceptions, encoding, intuitive associations, and interpreting of the machine integrated information to make decisions. We have validated FIR increased effectiveness and efficiencies at the most basic levels, by significantly reducing eye-saccades (eye-movements) during which the human perceptual system is blind; along with cognitive benefits as task-time reductions, and error-reduction advantages derived from reduced cognitive context-switching currently required during frequent, repeated, time-wasting, error-prone switching between the primary work area display and secondary Graphical User Interfaces multiple menus/modes/controls and separate-viewers. FIR further enables Intel Analysts beneficial focus, controlling, and inputting within the primary work area display via user selectable modular options for our Touch&Speak, Gesture&Speak, and/or Look&Speak again eliminating unnecessary traditional GUI interactions. These three innovations also automate intuitive selecting and placing 2525B Icons/Graphics, Annotations, Intel Hyperlinks, etc. FIRs IntelliMarks (Intelligent-Bookmarks) software modules enabling analysts to intuitively annotate/highlight, and effectively share objects/events of interest within ANY COTS/GOTS viewers of Intel content, and/or ANY displayed content. FIR Software-Development-Kit encapsulates/enables design-teams to implement our scientifically justifiable methods/guidance for improved design; BENEFIT: Our Fused Informational Realities (FIR modules are designed to be added onto existing Intel analysts workstations as well as manned and unmanned aircraft systems.Our modular add-on approach enables defense and/or commercial interface design teams to improve human-machine teams ability to fuse information.We envision new levels and types of shared human-machine teams understanding with faster more effective military results, for human-autonomous machine teams.Our team has already developed an integrated suite of scientific assessment tools designed to objectively assess combined human and system performance.Our suite has already been used to objectively assess two different GOTS common robotic controllers on military relevant tasks, to support a transition customers down-select. These tools provide a differential advantage to our team, and will be delivered as an essential part of our end-to-end solution for this Air Force project.We anticipate near-term operational uses by early adopters based upon successes we have already achieved in unmanned systems (unmanned aerial systems, unmanned ground vehicles, and networks of unattended sensors).Specifically, we have modularly enhanced existing government off the shelf common robotic controllers with our FIR innovations, our own next generation interfaces (see www.MirrorTrainingInc.com for our Wear Your Robot breakthroughs and press coverage) including successful scientific studies with military subjects and university students demonstrating benefits.Additional Intel analysts use cases, plus survival, lethality, and aircraft operational fusion and shared perception exemplars are included in the non-public document areas of this document.