Multiframe Super Resolution Leveraging Machine Learning

Situational awareness is an essential component in ensuring the safety of rotorcraft aircrew and aircraft from human error and hostile threats. Multiple optical sensors mounted on rotorcraft provide situational awareness in the form of high frame rate video for operators to view or automated algorithms to process. The utility of the information provided by these sensors is often limited by the resolution of the sensor. Multiframe super resolution is a software based approach to increase the resolution of an image by combining multiple low resolution images. Toyon Research Corporation is proposing the development of a multiframe super resolution algorithm that leverages recent advancements in image deblurring through machine learning. A deep convolutional neural network and general regression neural network will be used to classify and estimate the blur contained in low resolution images. Blur estimation along with accurate image registration will allow the information from the multiple low resolution images to be properly combined into a single high resolution image. This algorithm will increase the resolution and visual acuity of optical sensors onboard rotorcraft, allowing for a greater level of situational awareness and threat detection.