Sociolinguistic Information Filtering Tool (SIFT)

The Internet and social media are littered with bots, cyborgs, trolls, spam, deepfakes, shallowfakes, misinformation, disinformation, and other manifestations of purposefully manipulative content. Recent world events have highlighted the tangible and worrying impact of these phenomena on core social and democratic functioning. There is mounting evidence that bot and bot-assisted accounts, acting in often unwitting concert with the accounts of genuine social media users, can target individuals and groups with content designed to manipulate effectively at scale. These campaigns are fueled by a mixture of bots, cyborgs (bot-assisted accounts) and genuine individual human accounts. Combating such campaigns requires early and robust detection methods. The problem of these accounts is closely related to the more general problem that current social media users are unable to “fight back against the algorithms” to block not only manipulation and disinformation, but to have fine-grained control about the types of accounts and content they would like in their feed. We refer to the large class of bots, bot-assisted accounts, human propagandists, purveyors of disinformation and other user-determined objectionable feeds as “malicious accounts  This proposal for a Sociolinguistic Information Filtering Tool (SIFT) puts forth a vision of a desktop extension and associated mobile app initially based on advanced machine learning, natural language processing and network theory techniques, significantly refined by inputs from social science and user studies. Motivated by recent work understanding human information processing in the context of warning mechanisms, we propose to build these tools iteratively, with humans-in-the-loop and input from social science and detailed user studies (in Phase II). Through these studies, we will evaluate and refine sophisticated algorithmic approaches for account filtering and user-initiated bulk action toward automated, malicious, or otherwise unwanted accounts. In service to this goal, the proposed work offers four main innovations—powerful techniques that, to date, have not been brought to bear on this problem: Detection of online linguistic communities, leveraging QS-2’s extensive work on native language and speaker identification using advanced stylometric analysis. Use of spectral and game-theoretic clustering to determine frequency, content and coordination patterns. Employment of network-based machine learning approaches to predict the existence of and relationships between coordinated accounts, bots and humans, and other malicious accounts using the inputs of 1 and 2 above along with advanced network features. Development of a Twitter browser extension/app guided by extensive user studies and social scientific input.