DeepRL Sim-to-Real

PhysicsAI proposes to develop a framework for using deep reinforcement learning to create optimal training data from simulated imagery in order to train machine learning models with improved accuracy, increased robustness, and which support few/zero shot detection for rare objects. Specifically we plan to extend the current state-of-the-art by (1) formulating the task of generating an optimal sim-to-real training set as a Markov Decision Process; (2) developing several deep reinforcement learning approaches comparing methods based on policy gradients, deep Q-learning, and deep actor-critic reinforcement learning; (3) utilizing a test-bed with automated 3D scene generation combining CAD and terrain models, domain randomization techniques, and online learning to compute a local reward function; (4) demonstrating and benchmarking and our algorithms compared to baseline algorithms using a commercial high- satellite imagery dataset.