A Scalable Distributed Client Based Meta Search and Discovery Infrastructure

Web searching has become a ubiquitous and indispensable activity for a wide spectrum of human endeavors. The success of Google and its competitors depends on extensive software and network infrastructures and costly hardware. Federated search engines, such as Science.gov, WorldWideScience.org and ScienceEducation.gov also run on powerful, albeit much smaller, server clusters that connect to diverse government and non-government information sources. Many desirable federated search and discovery applications would require prohibitively expensive server farms and network bandwidth to provide classroom and home access to high quality educational content in DOE, NASA, NIH and other government scientific and technological databases for millions of students and researchers. Next generation federated search and discovery applications must implement economical and scalable solutions to accommodate new information services for very large numbers of potential users and information sources. The goal of this SBIR project is to find a highly scalable and affordable alternative to the server farms, network bandwidth and performance bottleneck by integrating the federated search application into the web browsers of individual users. Another important goal is to achieve better quality search results by developing a next generation distributed federated search technology with powerful semantic search and personalization capabilities. The Phase I project developed a proof-of-concept prototype running an advanced Web 2.0 federated search engine plugin inside the Firefox browser. The prototype has hybrid semantic search capabilities that are distributed between the federated search engine client and the remote knowledge base server. The Phase II work will research and implement robust federated search engine plugins and add-ons for the major web browsers. The Phase II research will also develop expanded semantic knowledge bases (e.g. for energy, health, education) and explore how the client-side federated search engine can optimally utilize remote knowledge base web services to support powerful semantic search functions that produce better search results. Commercial Applications and other Benefits as described by the awardee: The novel client-side federated search architecture, and the innovative semantic search capabilities to be implemented during the Phase II research, will open up significant new commercialization opportunities for very large scale applications in the education, health, science, technology and business markets in Phase III and beyond. At the same time, there will be important opportunities for this new technology to offer public benefits, especially in the form of free access to high quality government and other educational and health resources for tens of millions of students and the general public