STTR Phase I:Camera-Based Multimodal AI for Health Monitoring

The broader impact of this Small Business Technology Transfer (STTR) Phase I project will be to mitigate healthcare costs and challenges associated with the nation's growing aging population. The outcome of this project will be a system that enables holistic, accurate, and faster monitoring of elderly people that will 1) improve quality of life and health of senior citizens, 2) address the future nursing shortage, particularly in the geriatric field, and 3) reduce healthcare costs. The proposed technology can impact a wide range of public health applications that require continuous, non-intrusive health monitoring.This Small Business Technology Transfer (STTR) Phase I project applies real-time computer vision and sensor fusion processing for real-time continuous monitoring and analysis of health-related activities. In addition, this project takes an essential step beyond current solutions by analyzing seniors' group behavior and interactions with objects to determine whether they suffer from loneliness, behavioral disorders, or confusion states. This project utilizes multiple artificial intelligence algorithms to maximize the discriminative power to understand human micro-body motions and thereby create a training framework for personalized behavioral understanding and monitoring. It also proposes an innovative solution to leverage the results of computer vision as attention feedback to thermal sensors to enhance the accuracy and robustness of thermal processing against temperature variations. Furthermore, the project offers an integrative edge-based Internet of Things solution to protect citizens' privacy while providing real-time continuous feedback to responsible personnel. The proposed system offers a novel user experience while protecting personally identifiable information.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.