The Software Therapist: Usability Problem Diagnosis through Latent Semantic Analysis

Knowledge Analysis Technologies (K-A-T) and Virginia Polytechnic Institute and State University (Virginia Tech) will partner to fulfill this Research and Development effort. We propose an unprecedented suite of Usability Engineering software tools to bebuilt upon the conceptual foundation of Virginia Tech's User Action Framework (UAF). We will use K-A-T's proprietary Latent Semantic Analysis (LSA) methods and software tools in Phase I to validate and refine the UAF. We will also use LSA as theunderlying analysis engine for the software tools, which will provide support for usability problem extraction, analysis, diagnosis, plus links to related literature and prescriptive solutions. Tool prototypes will be developed in Phase I; commercial gradedevelopment of the suite is proposed for Phase II.The major research thrust of Phase I is exploration of LSA techniques for free text analysis, trained on the literature of Usability Engineering and a significant library of usability problem reports, to validate and tune the taxonomic structure andcontent of the UAF. Because no standardized vocabulary exists for usability engineering, simple keyword methods cannot reliably classify problem reports. In contrast, LSA can provide highly reliable measures of semantic similarity between texts even whenthere is no keyword overlap. Phase I will deliver an improved and validated UAF as well as specifications and prototypes for the Usability Engineering tools proposed for Phase II. The potential benefits of both deliverables are immense. The HCI field is in dire need of such a toolset for both practitioners and students. Software developers in commercial, government, military and academic settings are spending significant dollarsin usability testing. However, the field lacks strong tools to link discovered problems with known solutions. The UAF provides the theoretical and conceptual basis for an engineering support system that will classify a dynamic and growing database of Lessons Learned usability problems and solutions. LSA provides an intelligent information discovery and retrieval system that will allow both novice and expert usability engineers to succeed in extracting appropriate solutions and knowledge from thedatabase. The return on investment in this software for both (1) engineer time savings in solving specific problems and (2) institutional development of the knowledge base will be very significant. We anticipate a market for this product from softwaredevelopment organizations (commercial and government) as well as from academic programs for HCI and usability engineering.