ACCESSIBLE WEB AUTOMATION FOR PEOPLE WITH VISION IMPAIRMENTS

This project builds upon previous research to develop and deploy the Assistant, an enabling technology for accessible web automation for people with visual disabilities. Web automation, as embodied in the Assistant, automates many of the repetitive low-level actions such as form filling and activating controls (such as clicking on buttons and links) that one has to do in typical browsing tasks such as online shopping, banking, course registration, bill payments, reservations, etc. While these tasks are easy for sighted people, they pose considerable difficulties for people with visual disabilities, especially those who are blind. The Assistant dramatically improves the speed and efficiency with which individuals who are blind or have low vision can do the aforementioned online tasks. The Assistant is driven by a predictive model coupled to an intuitive user interface. A unique feature of the Assistant model is its ability to automatically learn from the history of browsing actions. The Assistant utilizes this model to predict and automate future browsing actions. The interface suggests appropriate browsing actions and enables users either to follow the suggestions or bypass them and browse normally.