Source Selection Optimization

A major scalability problem of federated search engines, knowing which sources to search, occurs when a hundred or as many as a thousand or more sources are available to the researcher. It is not a good idea to search all possible sources for every query, as this places a heavy burden on the search framework as well as on the content provider. Placing the burden of source selection on the researcher is not a good solution, as he may not know which sources are the best. This project will design and develop a prototype algorithm and application that automatically selects the most appropriate sources for a given query. The approach is based on gathering and performing real-time analysis of search statistics, i.e. query terms, sources searched, and relevance of results. Commercial Applications and other Benefits as described by the awardee: The new search solution should improve the utilization of available sources, as researchers may only be aware of sources in their specific field of study (and perhaps not even all of those). The capability also would support the cross-fertilization of ideas from different disciplines, promoting global discovery and the diffusion of knowledge. The new search solution should be of interest to organizations that have large numbers of sources to search, especially where it is impractical to search all the sources in parallel and where important sources must not be missed.