Soft Biometrics-based Annotation and Retrieval for Large Scale Face Database

We propose to design, develop and evaluate a system for retrieval of face images using natural language-like descriptions. Our approach is largely guided by how humans perceive similarity between faces and incorporates a combination of soft biometric descriptions, local and global feature descriptors and manifold-based similarity measures and machine learning techniques. Using a reasonably-sized training face data set from widely available sources including the web, we design a series of machine learning techniques for annotating the faces using soft biometric features. A much larger face data set containing millions of faces is then annotated using non-Euclidean distance measures defined on face manifolds. Given a new face in an image or a video, we will design techniques for fast and efficient retrieval of similar faces. In Phase I, we will design the modules for automatically annotating a large face data set and a simple interface for retrieving faces similar to the given face. In Phase II, we will scale the approach to handle several millions of faces and demonstrate a prototype for fast matching of faces.