Assessment of Asymmetric Social Indicators using Non-Verbal Vocal Cues

The objective of this proposal is to develop, implement and evaluate a set of accurate and robust algorithms specifically designed to capture and measure multilateral social interactions automatically through audio-based methods based on asymmetric data design. We propose to accomplish this by: 1) Developing a novel method that uses multiple microphones interfaced with a smartphone and placed on a single participant to detect, segment and identify participants in a conversation using non-linguistic methods; and 2) Measuring interactions using conversational dynamics. We propose to evaluate our asymmetric design under controlled experimental laboratory scenarios with contextually relevant noise (i.e., explosions, traffic and street noise, etc.) to demonstrate robustness of the proposed audio methods to relevant military and medical scenarios. Finally, we aim to link the smartphone to a web-based platform that enables secure graphical visualization of transmitted social interaction data from a remote setting in a near real-time manner.