SBIR Phase II: Large-Scale Behavioral Analysis Utilizing Convolutional Neural Networks and Its Application to In-store Retail Marketing

The broader impact/commercial potential of this Small Business Innovation Research (SBIR) Phase II project will be to enhance infrastructure for research and education by exposing a previously unavailable dataset: fine-grained human interaction with a physical environment. Humans are always building on and shaping the world but there is little hard data to further examine the effects this has. Beyond retail, this technology could affect how teachers layout classrooms, how disaster workers provide relief, or how factories keep their workers safe. The subtle physical details that affect humans everyday will be understood and investigated in ways not possible without the proposed system. This technology will benefit society by increasing economic efficiency because retailers can meet customer needs more easily. This translates into large potential commercial value because of the size of the retail market and because retailers are under pressure to deliver differentiated customer experiences that customers cannot get online or at big-box stores. Such experiences are enabled by understanding the customer at a much deeper level which in turn is enabled by the technology proposed here. This Small Business Innovation Research (SBIR) Phase II project uses sensor placement models, statistical methods, and face recognition to fully realize the commercial potential resulting from the Phase I development of a video analytics system for understanding human behavior. Research in video-based behavior recognition has seen renewed excitement because of deep learning but current work only addresses pieces of the problem. Critical missing components are robust face identification and registration, a process to install as few cameras as possible while maximizing the viewable area of a store, and video analysis results that are meaningful when combined with other data such as retail transactions. The research objectives of extracting biometric data (such as facial features) from video, automatically computing the optimal positions of cameras, and correlating behavior metrics with business operations are essential to improve the retail experience for shoppers and make the developments undertaken in Phase I more commercially relevant to potential end users of the technology.