Exceeding Limits Beyond Ordinary Wearables (ELBOW)

The project aims to create a framework for real-time wearable sensor analysis that can be used to monitor warfighter health and readiness. There are two parts to the work: Framework and Model. The framework will be built on a smartphone and provides the flexibility to add any number of Bluetooth Low Energy wearable sensors and/or internal smartphone sensors while maintaining low energy consumption. The framework will be modular so that different data analysis models can be swapped out and experimented with. The models will include different artificial intelligence and machine learning algorithms that can recognize human activity and anomalies to be used for personal health determination.  The project differentiates itself from others by using IMU pose-based multi-wearable data instead of raw sensor data from multiple wearables. This type of data is of particular interest because although there is plenty of research on camera-based pose data, there is very little on IMU-based pose data. The use of an integrated full-body tracking system corrects noise from individual sensors through corrections found from sensor fusion of full-body data. The pose data is also used as a verification method for the data itself as pose data is easily visualized and understood by the human eye which provides a convenient way to label the information. The labels will consist of hierarchical structures to help the machine learning algorithms better differentiate between activities.  The human activity recognition models include experimenting with full-body pose data on the three types of categories: Template, Generative, and Discriminative. Specifically, the tested algorithms are Dynamic Time Warping, Hierarchical Hidden Markov Model (HHMM), and the Random Forest Classifier. To verify that the models work as intended, work will be performed to automatically create poses that can be properly classified by the model. This will be done with Generative Adversarial Networks as their main goal is to find the important features in data that lead to the classification. This way, a large amount of properly labeled data can be artificially created without having to do manual data collection.  The anomaly detection models are split into two parts: i) anomaly within activity and ii) anomaly within routines. Within an activity, anomalies are found through outlier analysis such that data points that differ too greatly from the usual way the person would perform the activity are flagged. Within routines, the anomalies can be found using an HHMM. As all the state transitions have a dynamic probability, anomalies are transitions that occur even though they have a low probability of occurring. Similar to human activity pose generation, verification is done by simulating anomalous data within activities and by simulating anomalous routines. As before, these outputs can then be manually checked using the pose data viewer.