VMS: Deep Reinforcement Learning for High-fidelity Vehicle Motion Simulation

NGA seeks to incorporate Artificial Intelligence (AI) and Machine Learning (ML) into Intelligence, Surveillance, and Reconnaissance (ISR) missions to capture fleeting targets. As such, a large number of dynamic scenes with accurate target motions and behaviors will be needed for training and performance evaluation. The traditional microscopic model-based approach for vehicle activity simulation is unable to produce enough fidelity for the training and the evaluation of ISR tracking, analytics, or collection strategies. Intelligent Automation Inc. (IAI) proposes to develop a high-fidelity Vehicle Motion Simulation (VMS) system that includes a framework for SUMO (Simulation of Urban Mobility) based traffic simulation and validation with real-world roadway sensor data coupled with a deep reinforcement learning (DRL) model for activity simulation of target moving vehicles. In Phase I, we successfully demonstrated the feasibility of our proposed VMS framework and we have made significant progress towards the ultimate goals of this project. In Phase II, we plan to extend the scope of our simulation and reinforcement learning framework and build a full-fledged prototype software tool that addresses NGA’s needs.