Image Representative and Retrieval Using Image Context Vectors

The burgeoning growth of image and video databases demands fast and effecitve methods for representing and searching image collections. Most current approaches depend upon labeling images with text keywords. Manual labeling is labor-intensive and difficult, because one must predict which parts of an image will be of future interest, as well as what query terms will be used for the retrieval. This research seeks to develop a fully automatic method for representing images that allows query by images, subimages, or terms. The Context Vector appraoch, previously developed for document retrieval in ARPA's Tipster Text Processing Initiative, will be generalized to Image Context Vectors (ICV's) that can handle the special requirements associated with image retrieval. For example, ICV's are designed to be robust with respect to changes in size, position, and orientation of objects within an image, as well as partial occlusions of objects. Phase I will provide a proof-of-concept for the ICV approach as applied to a database of 200-500 images. Phase II will demonstrate effective performance on video images and ona large image database, will experiment with advanced representation techniques, and will explore neural network learning algorithms for adaptive improvement of initial queries. Anticipated Benefits: A successful project will allow automatic indexing and flexible searching of image databases and video libraries; queries can be images, snippets (subimages), or terms. Image Context Vectors may lead to fundamental advanceds in Image Understanding by catalyzing new approaches to image representation.