Real-time Forensics of Online Influence within Visual and Audio Content

Foreign influence operations represent a persistent threat in the online information sphere, but remain particularly difficult for information operators and intelligence analysts to detect.  Even with the benefit of hindsight, researchers still struggle to understand and forecast the full range of these influence operations, and we risk falling even further behind as the information environment shifts from primarily text-based content towards an increasing volume of visual and audio data. To address this gap, Stottler Henke proposes to develop the Forensic system, which will deliver a substantial leap forward in the identification of new behavioral analytics that can detect and classify information maneuvers within visual and audio content.  Forensic will address this challenge by: 1) codifying simple heuristics that do an imperfect job at identifying influence maneuvers from signals within visual and audio content; 2) leveraging those inexact heuristics to train a behavior analytics model without needing any individually labeled data; and 3) generating descriptions and visualizations that explain the detected online behaviors and mechanisms of influence.  This effort leverages our team’s past work and experience in the areas of open source information analysis, event/threat forecasting, and social and behavioral science.