Machine Learning for Simulation Environments

Areté and the Machine Learning for Artificial Intelligence (MLAI) Lab at the University of Arizona (UA) will complete the development of an interactive scenario building tool to generate realistic simulation content in real-time for use in training simulators for periscope operators.  Areté calls this tool RealSim.  The novel capability is created by combining the latest advances in generative adversarial networks (GANs), texture generation, ocean modeling, the Unreal Engine's simulation capabilities, and Areté's experience in building periscope technologies.  Areté will deploy this technology as a suite of algorithms useable in any simulation application and demonstrated in RealSim. In Phase I, Areté developed a prototype scenario generation tool and demonstrated various techniques to bring enhanced realism to scenario imagery.  The Unreal Engine (UE) is the core platform used to create the scenario generation tool. UE will continue to be used as the mechanism for scenario development and demonstrate Areté 's ability to bring realism to simulated imagery. Areté's initial design was to post-process simulation imagery after scenario rendering.  However, lessons learned during the Phase I program have led to an update in bringing realism to dynamically generated scenarios.  A new processing flow replaces the initial step proposed for Phase I.  There are four differences from Areté 's initial design.  First, in place of a tiered-GAN approach, there is a set of specialized GAN models that focus specifically on different regions of the imagery: land, sky, and ship textures.  Second, sophisticated ocean models will accomplish realism for the water.  Third, all enhanced realism modules will be implemented as external plug-ins to the UE Renderer, and finally, RealSim will have five options for output (i.e., imagery and video) directly from the application.