Labeling of Text to Train Machine Learning Models to Improve the “E� in the USAF’s Processing, Exploitation, and Dissemination Systems

The USAF relies on automated text processing systems powered by machine learning to support its Processing, Exploitation, Dissemination mission. The effectiveness of these systems are dependent on domain specific, relevant, and current labeled training data. Yet often, due to to a variety of human, data, and machine challenges, training data sets do not keep pace with the semantic evolution and expansion that is inevitable in a dynamic PED environment. Thresher proposes a feasibility study in conjunction with the 480th ISR Wing to examine the extent to which QuickCode--commercially available software for quickly and transparently labeling text for training data--can help analysts get better results from the machine learning systems they use to process text. The feasibility study will use publicly available data in at least two languages to demonstrate the speed, transparency and accuracy. In addition, the study will use structured approaches for mapping user needs with a particular focus on requirements for integrating QuickCode into an analyst’s workflow. QuickCode makes use of Thresher’s patent-pending multilingual label recommendation method built on an open source, machine learning stack, originally developed at Harvard to help researchers find code words in Chinese social media and augmented with funding from DARPA.