Video Information Summarization, Captioning, Analysis, and Rank Ordering (VISCARO)

Military and open-source videos contain a wealth of information about entities, concepts, and events of interest to U.S. and coalition forces. Analysts have access to these video sources but are severely limited in exploiting the information contained within due to manpower and technical limitations. To fully understand the relevance of a video to mission objectives, analysts must watch each video in its entirety. This manual, time-consuming process is laborious, tedious and prone to errors. Addressing this challenge requires a multi-disciplined approach. Unlike most data sources, video data is multi-modal, with relevant information contained in the video stream, audio stream, and text that is embedded into the image frames. Most significantly, none of these individual modes typically describes the context and meaning of the video completely. It is the combination of all of them, fused together, that fully conveys the true story provided by the video. To address this challenge, the DECISIVE ANALYTICS Corporation (DAC) team proposes to develop the Video Information Summarization, Captioning, Analysis, and Rank Ordering (VISCARO) system. VISCARO will fuse data from all modalities into a common semantic data model to enable automated generation of text-based summaries, meaningful natural language search of a video corpus, and video recommendation.