Situation Understanding and Reasoning Evaluation (SURE) Toolkit

The Army has engaged in a modeling and simulation effort to develop advanced technologies and tactics, techniques, and procedures (TTPs) supporting the transformation to network-centric warfare (NCW), which has culminated in the Modeling Architecture for Technology, Research, and Experimentation (MATREX). Within MATREX, the Command, Control, and Communications Human Performance Model (C3HPM) defines operator behavior using a hybrid architecture integrating task network models and rule-based inferencing engines to model both decision-making behavior and task performance. Enhanced usability and analysis of C3HPM performance faces two major challenges: 1) overcoming the complexity of developing high-fidelity models through tools supporting the automated generation of behavior models; and 2) developing analysis tools to identify sources of error in the C3HPM’s cognitive processing skills, particularly those associated with situation understanding (SU), a key component in skilled decision-making. We propose to address these challenges through the development of the Situation Understanding and Reasoning Evaluation (SURE) toolkit, comprised of the following components: 1) an ontological process and tools supporting rapid specification and generation of behavior models; 2) an automated pre-execution SU analysis tool assessing model performance in a stand-alone verification and validation process; and 3) an automated post-execution analysis tool for evaluating SU performance within the MATREX environment.