SBIR Phase I: Large-Scale Behavioral Analysis Utilizing Convolutional Neural Networks and Its Application to In-store Retail Marketing

The broader impact/commercial potential of this Small Business Innovation Research (SBIR) Phase I project is to develop an infrastructure for exposing and interpreting a previously unavailable dataset: fine-grained human interaction with a physical environment. Humans are continuously building and shaping the world but there exists little data to examine these effects. Beyond retail, this technology could affect how teachers layout classrooms, how disaster workers provide relief, or how factories keep their workers safe. The subtle physical details that affect humans everyday will be understood and investigated in ways not possible without the proposed system. This technology will benefit society specifically by improving the economic efficiency of retailers and broadly by increasing scientific understanding of how humans interact with their physical environments. This Small Business Innovation Research (SBIR) Phase I project uses a neural network and generic 3D scene reconstruction in combination with low-cost, wireless cameras to model an environment with accurate object classification and spatial relationships. Recently, neural networks have proven adept at a variety of image classification tasks but their applications in video classification, namely for human actions, have been less explored. 3D scene reconstruction has made similar advances, progressing from images to videos but has always required some prior knowledge of the physical scene. Within the past year, multiple groups have proposed methods for completely generic scene reconstruction from multiple view cameras. Finally, energy harvesting methods for devices such as cameras and wireless transmitters have been demonstrated to be feasible in laboratory experiments but have not been incorporated into commercial products. In this project the two computer vision algorithms will be developed in parallel with camera hardware so that the software and hardware systems may be integrated and demonstrated by the end of the project.