SBIR Phase I: Using Deep Learning and Action Recognition to Automatically Digitize Human Actions at Scale: Putting Workers at the Center of the Next Industrial Revolution

The broader impact/commercial potential of this project is to enable the automatic analysis of human motion using visual information gathered at high frequency. Within the manufacturing context?which represents 11% of US GDP [Bureau of Economic Analysis]?the collection and interpretation of this new set of large-scale time-and-motion data enables dramatic improvements in the understanding of assembly processes and optimization of human productivity. In addition, manufacturers can use this system to flag process errors or deviations?ideally in real time?allowing for mitigation before the deviations propagate further down the value chain. Just these two capabilities enable manufacturers to avoid costly rework and recalls, improve worker accuracy, discover new opportunities to optimize processes and generate revenue generation, and flag worker safety issues. As a result, workers and management become aligned around shared goals of efficiency and competitiveness?reducing fears of automation while dramatically improving productivity and possibly protecting jobs. This Small Business Innovation Research Phase I project will improve the robustness of a prototype deep learning back-end for automatic action recognition from a stream of video data. While object recognition is now commonplace, action detection?e.g., inferring the behavior and intentions of actors and objects over time?has not yet been solved or commercialized. In fact, it is still an active research area. Solving this problem requires overcoming technical hurdles in industrial settings that include changing actors, lighting conditions and camera perspectives; manual labelling of large volumes of video data; transmitting large volumes of data to and from the cloud; accurately inferencing with high levels of confidence; and developing intuitive human/system interfaces that may, in the future, include unconventional channels such as AR/VR, text-to-speech, haptic feedback, amongst others.