Realistic Ocean Water for Simulator Environment Augmentation

This is Lynntech’s response to the exploratory STTR solicitation titled “Machine Learning for Simulation Environments” to improve the realism of periscope simulators, by, as much as possible, augmenting the front-end display of the Navy’s Submarine Multi-Mission Team Trainer (SMMTT). SMMTT incorporated physics-base elements but produces pristine low-res image frames.  Our Realistic Ocean Water for Simulator Environment Augmentation (ROW4SEA) system will be implemented using a machine learning training and a style transfer generative neural network. The machine learning resource of advanced dynamic ocean environment and water simulations with 360-degree views provided by our academic partner Jerry Tessendorf of Clemson University, and will be able to simulate different sea states. Jerry Tessendorf is an Academy Award-winning professor (2008 Academy Technical Achievement Award for fluid simulation technology, exemplified in many feature films including the 2012 film Life of Pi, which received the 2013 Academy Award for Best Visual Effects) of visual computing from Clemson University who revolutionized the use of fluid simulations in motion picture computer graphics with his seminal 2001 work “Simulating Ocean Water” will join the Lynntech team by providing advanced dynamic ocean water simulations with innovative 360-degree panoramic views. The Lynntech Intelligent Systems group in turn will apply their experience in developing generative networks to realize a video Stream to Stream with 360-degree views (S2S-360) style transfer deep neural network which will be trained on the machine learning resource developed by Dr. Tessendorf’s team which will also include the corresponding low-resolution segmentation images with relative depth information as well. The S2S-360 style transfer network would be an innovative extension of the pioneering vid2vid progressive generative adversarial network (GAN) in two ways: (1) allowing style transfer to be applied to video streams in near-real time and (2) allowing for style transfer across 360-degree views. Furthermore, the S2S-360 will involve not only framewise pixel segmentation maps but also a depth channel to allow for dynamic elements to possibly obscure other farther away elements in the scene. The ultimate goal is that the machine learning resource will be used to implement the front-end augmentation tool for the existing SMMTT system, enabling more realistic training experiences, more challenging contact finding and ultimately more effective training outcomes. The relevant prime contractor for SMMTT, Lockheed Martin – Rotary and Mission Systems (LS RMS) C6ISR SBIR Team, has also issues a Letter of Intent to support Lynntech in this effort begin, with information relevant for Phase II planning as early as the Option period.