SBIR Phase II: Filmmaking for Everyone: Computational Video Editing

The broader impacts of this Small Business Innovation Research (SBIR) Phase II project enable improved user-generated video content. Video online platforms have radically changed how people communicate, learn, and inspire. Unfortunately, many potentially inspiring videos are lost or never shared due to an inability to edit them into compelling vignettes. Most tools for editing video are expensive, difficult to learn, and time-consuming. This project’s research enables consumers or nascent businesses to make polished, professional videos with a single phone click through the use of computational cinematography and techniques from deep learning and artificial intelligence (AI). The delivered software solution will produce high-quality edited footage within minutes, compared with a human editor requiring hours. This project combines the analysis of video using computer vision with editing algorithms to empower new creators to participate in this fast-growing medium. This Small Business Innovation Research (SBIR) Phase II project will produce AI-powered software for automatically editing raw video footage into quality short films on a mobile phone platform. The proposed project will integrate advanced computational video manipulation, computer vision, and audio recognition. The prototype AI editor will select relevant content from source footage and synchronize it to music, using only the restricted computational resources of a typical mobile platform. The AI makes decisions based on the video content, the music content, and narrative editing styles learned from a large dataset of similar films. This project will deliver AI-based editing technology that trims and arranges input footage based on the spoken dialogue in the input videos. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.