Multimodal Information Perceptualization for C4ISR Systems

Current information visualization (IV) techniques primarily focus on visual presentation of data. While these techniques aid human system integration (HSI) design, more advantageous designs may be possible with multimodal information perceptualization(MIP). This new design concept displays information through multimodal (e.g., visual, auditory and haptic) channels, thereby expanding current IV concepts. By capitalizing on a user's available information processing areas and accounting for varying tasksdemands, particularly in time-critical C2 environments, MIP design holds great promise. This Phase II effort will take the MIP design guidelines identified in earlier work (Phase I) and integrate those pertinent to C4ISR environments into an interactiveMultimodal Information Design Support (MIDS) system that provides intelligent HSI advice to multimodal system designers. To further optimize manning of C4ISR systems, it is critical to assess capabilities of warfighters, and identify individuals with thehighest capacity for multimodal information processing. The proposed work includes development of a Tool for Information Processing Capacity Assessment (TIPCA) of individual warfighters, which will provide objective assessments of individual informationprocessing capacities. Together, these two tools (MIDS and TIPCA) will enable designers to provide enhanced human/system integration with C4ISR environments through efficient multimodal display design and appropriate manning allocation. Development of aninteractive, multimodal information design support system and a tool for information processing capacity assessment to aid designers in assessing and modifying multimodal human-computer systems offers wide utility to the HCI design field in general and tomilitary system design in particular. These tools will allow not only evaluation of current C4ISR-type systems in terms of SA and information overload, but will also provide design guidance for new systems which are in a concept development stage. Inaddition, the products from this research extend beyond system design to also focus on the operator's ability to succeed in a complex, time-critical multimodal information environment. By combining validated system and operator assessment tools, designerscan enhance human/system integration of C4ISR systems and create a more streamlined, efficient work environment.The proposed MIDS system and TIPCA could be employed for C4ISR tasks performed in diverse training systems, hence facilitating design simplification and procedural consistency for military training systems. The estimated market size for a productresulting from a funded Phase III effort is in the range of $1.5-$3.0 million, based on comparisons to other design software in the HCI field.