Framework for Integrating Cognitive Models into Virtual Environments

We propose to develop a framework for linking cognitive models with virtual environment simulations of the sort that are typically used in military training applications. The framework will be designed to accommodate a variety of different cognitive models and simulations. It will supply a cognitive model with sensory information obtained from a simulation, and enact cognitive model motor and other commands in the simulation, while imposing human-like limitations on sensory information processing and control. The framework will be based on a formal analysis of requirements, including surveys of candidate simulations, cognitive architectures, and human performance data. It will include its own graphics and physics engines so that, where necessary, it can supplement an attached simulation in order to present cognitive models with complete and consistent phenomena. It is based on portable software and distributed processing middleware, allowing the framework to operate on and be accessed from many common computing platforms and language. And it includes logging, replication and monitoring mechanisms to facilitate experimentation. BENEFIT: The framework we propose to develop will provide benefits for both the simulation and cognitive architecture communities. The framework will allow simulation developers and users to more readily incorporate intelligent teammates, opponents, and teachers into their virtual environments, thus supporting improved training and evaluation. And it will give cognitive architecture researchers access to a much richer set of virtual environments, provide more human-like inputs for cognitive models, and reduce integration and experimentation cost. In the longer term, the framework will enable improvements in the cognitive architectures and models used in commercial applications such as intelligent tutoring, robotics, and computer games.