Common Unifying Representation Embeddings (CURE)

Assured Information Security, Inc. and Tufts University, propose CURE, an effort that will perform novel transfer learning research to enable the practical application of Reinforcement Learning (RL)-based Artificial Intelligence (AI) for widescale use in both military and commercial systems. CURE is a research effort to investigate and develop novel and effective transfer RL techniques based on world-models that learn to disentangle what is common and what is not between domains in the form of common embedding spaces. This approach is inspired by recent work in image and style transfer, where learning these common embedding spaces has produced strikingly effective transfer and generalization of images from one style domain to another. Unlike traditional transfer RL approaches that attempt to learn how to generalize and adapt individual policies from one domain to another, CURE will develop methods to learn disentangled embeddings, which are latent models of two domains that explicitly represent what is shared between the domains and what is not. By modeling what is shared between domains, instead of what is shared between policies, CURE will enable broader and more effective transfer through improved domain adaptation and cross-domain planning. CURE will use these disentangled embeddings to develop transfer-aware policy learning algorithms that actively use knowledge of shared embeddings to effectively boost transfer and generalization across domains. Additionally, CURE will use the shared embeddings and examples of positive and negative transfer to train a metric model to predict transfer success. Such a metric could provide a principled and effective method for determining the degree to which two domains can transfer policies from one to another. Finally, CURE proposes novel research building upon our team’s pioneering work in curriculum learning for effective transfer in complex multi-task domains.