An Analysis of the Performance Bottlenecks in the Federated Search Information Flow

Performance bottlenecks in federated search limit the benefits of the technology to DOE researchers, to other scientists and to the public. We will assess the performance bottlenecks of Deep Web Technologies' federated search technology in Phase I and make performance improvements in Phase II. We performed a rigorous assessment of the information flow in our federated search technology. We identified a number of bottlenecks and assessed the impact of each to prioritize the performance improvement work. We also prototyped technology to improve certain performance bottlenecks. We will design and implement a number of technologies to remove major bottlenecks. The technologies will improve the speed of data access and storage, networking, web access, computation, and connector work. Commercial Applications and Other Benefits: The future of federated search lies in the ability to cast a wide net over an ever-increasing body of scientific information. It will become more feasible to build large-scale federated search applications for the federal government that simultaneously search thousands of sources.