Real-time Forensics of Online Influence within Visual and Audio Content

Foreign influence operations represent a persistent threat in the online information sphere but remain particularly difficult for information operators and intelligence analysts to detect.  To address this gap, Stottler Henke proposes to develop Forensic, a system to rapidly assess the utility of audio/visual features as indicators of influence maneuvers, particularly when labeled data of those maneuvers does not exist.  First, Forensic automatically extracts a wide range of features from visual and audio content and facilitates the creation of simple labeling heuristics that do a “good enough” job at identifying influence maneuvers from those features.  Then, the system leverages those “noisy” heuristics to train a strong predictive model for detecting influence maneuvers—without any individually labeled data.  Lastly, Forensic generates explanations and visualizations of the detected online behaviors and mechanisms of influence.  Demonstration of the essential elements of our system concept during Phase I has established a strong technological foundation enabling our team to set very aggressive objectives for Phase II.  By applying the lessons learned and software developed during Phase I, we will demonstrate the full capabilities of Forensic by the end of Phase II, setting the stage to transition the technology into one or more civilian or DoD programs.