Model Zero - Reinforcement Learning for Policy Transfer into High-Fidelity Environments

Recent demonstrations of super-human performance leveraging reinforcement learning has shown the power of Artificial Intelligence (AI) for solving high dimensional complex problems through long-term decision making. However, most reinforcement learning approaches require hundreds of millions or even billions of training samples to achieve high performing policies. In this paper we propose a novel model based reinforcement learning algorithm call Model Zero that constructs a world model learned from a lower fidelity simulation and transfers the world model into a high fidelity simulation. Model Zero leverages its existing knowledge learned from a low fidelity simulation and continues to learn a more accurate world model after being transferred into the high fidelity simulation. A key advantage of using our model based reinforcement learning approach is that the methods are general and can be applied to various deterministic  high-fidelity physics based environments where training samples are computationally expensive to obtain.