SBIR Phase I: Virtual Interview Trait Estimation Combining Speech and Touch

The broader impact/commercial potential of this Small Business Innovation Research (SBIR) Phase I project is in improved efficiency in labor markets by enabling faster, more accurate assessments of the interpersonal traits ("soft skills") of job candidates, employees, and students. The project develops methods for accurate automated screening interviews that can save time, effort, and money for both employers and job seekers - delivering a more efficient candidate selection process for U.S. companies and public agencies that conduct hundreds of millions of screening interviews with job candidates each year. This project is focused on performance assessments of attitude, energy and social communication, specifically as these qualities are needed for selection of people for cooperative roles in employment settings. The project's applied research will refine (and may reconceive) the behavioral definitions of traits that are valued in employee selection because they are held to be predictive of success in work settings and in occupational training. Resultant refined measurement tools may enable advances in applied fields like industrial psychology and provide more precise variables for use in basic studies of the neurological correlates of emotion, social perception, personality, and mental disorders. This Small Business Innovation Research (SBIR) Phase I project designs, develops and fields a prototype app that conducts an automated Virtual Screening Interview (VSI). The VSI app is an experimental instrument that probes job-relevant traits in speech and touch responses. In recent decades, algorithms have estimated people's affect, sentiment, and emotions as observed in facial expression, text content, tone of voice, speech rate, gestures, posture and other body language. Among these, speech prosody, linguistic content, and facial expression have proven to be strong indicators of emotional valence (attitude) and psychomotor activation (energy level). VSI system research will identify measurable traits that support rapid learning and successful performance in activities and occupations that require teamwork or social skill. The VSI app collects performance data (voice and screen-touch tracks) on mobile devices to investigate novel combinations of features from speech and motion that accelerate and improve trait measurement. The project will generate new knowledge about how the relationships between speech and movement can match expert evaluations of people in work groups and in social settings. Measurement technologies developed in the project may also be applied in education and industry to improve the efficiency and effectiveness of instruction and training.