Automated Diagnosis of Usability Problems Using Statistical Computational Methods

Poorly designed software and hardware systems are inconvenient at best; at worst, they are deadly. Usability evaluation techniques are designed to detect the potential for disasters and inefficiencies in poorly designed software. However, usability analyses are prone to failure due to conflicting usability frameworks, incomplete usability reports, and inadequate analysis tools. Aptima and the Center for Intelligent Information Retrieval (UMass Amherst) have partnered to develop AutoDiag, an Automated Diagnosis System of Usability Problem Reports, and bundle it with a digital video screen capture application and a data logging application. Aptima will integrate these to create UPDATE, a Usability Problem Data Analysis Technology. The system will consist of an interface to elicit better usability reports, a text analysis engine that classifies these reports (or existing usability reports), and an analysis interface for diagnosing usability faults and identifying trends. The proposed effort builds on Phase I progress, which includes: development of a usability taxonomy by practicing usability professionals, application of that taxonomy to usability reports, competitive evaluation of text classifiers, and conceptual design of UPDATE. The proposed team consists of Aptima, a leading human-centered engineering firm, and the Center for Intelligent Information Retrieval, the pre-eminent laboratory for text classification research.