Clarifai Proposal - Automating tilt and roll in ground-based photos and video frames

For this proposal, Clarifai would utilize internal expertise in computer vision and deep learning to pursue a CNN to camera orientation estimation. Specifically, we would adapt existing Clarifai models to extract image features and use these features as input to a camera orientation regressor. The horizon filter information might be combined with information from the encoder to produce a joint embedding and this could inform the prediction of pitch, roll and their associated uncertainties. Alternatively, image based filtering could be used to generate camera pose, and then this information could be modulated by the horizon filter. Another item work considering is the use of transformers in the network. Transformers are used in language processing to account for the fact that order matters (but without introducing the complexity of training networks with memory, e.g. LSTMs). Their use in image processing is growing and seems especially appropriate for the pose estimation problem. The Clarifai team has extensive experience implementing and training  CNN based computer vision models, understanding the potential pitfalls that arise, and can diagnose issues when the networks are not properly converging. \n\n\nClarifai is the industry's leading independent artificial intelligence company headquartered in New York City with offices in San Francisco and Washington D.C.  Founded in 2013 by Dr. Matthew Zeiler, a foremost expert in machine learning, Clarifai has been a market leader since winning the top five places in the image classification challenge at the ImageNet 2013 competition. Clarifai provides powerful AI and Machine Learning (ML) image and video recognition solutions, built on the most advanced machine learning platform, empowering businesses all over the world to build the next generation of intelligent applications. Our team provides image and video recognition solutions to the Department of Defense, Intelligence Community, and commercial customers like West Elm, Trivago, and Opentable. In November 2019, Clarifai was named a visionary leader in The Forrester New Wave: Computer Vision Platforms, Q4 2019 report. Also, in November 2019, Clarifai was ranked 71th Fastest Growing Company in North America on Deloitte’s 2019 Technology Fast 500.