Vision with a Purpose: Inferring the Function of Objects in Video

On Phase 1, we developed an approach to functional object recognition that learns functional models from video tracks in an unsupervised fashion. On Phase 2, we will expand the algorithms and concepts developed on Phase 1 into a comprehensive prototype system that performs functional object recognition across a wide variety of object classes. The system will be robust against track fragmentation and errors in observables, and it will incorporate local scene context and object interactions. Additional object classes can be added with little effort, as unsupervised learning will reduce or remove any requirement for labeled training data. The developed capability will be integrated into VIBRANT, our VIRAT system, as a new content descriptor module that can be indexed and queried.