Automating Scalability of Federated Search in a Cloud Computing Environment

The ability to thoroughly search large numbers of scholarly information sources is vital to DOE researchers to accelerate scientific advances that address the nation's pressing energy and security problems. While large-scale federated search deployments provide the best hope of making research highly efficient and effective, there are barriers to scaling. Cloud computing is an important approach to consider to achieve scalability and its adoption in government is being urged by the Obama administration. We aim to prove feasibility of enhancing our state-of-the-art federated search technology to operate in a cloud. In Phase I we will assess, architect and implement some features of federated search, in particular, features that make best use of cloud computing, e.g., scalability, load balancing, failover, security and stability. Phase II will extend the scope of the Phase I work to build a federated search system optimized to run in a cloud. Commercial Applications and Other Benefits: Researchers and the public will benefit from a simple-to-use search application that has tremendous power to reach large portions of the "Deep Web" of scientific information that Google can't access. Much of that information is in foreign languages. The scalability achieved through cloud computing will allow for tens of thousands of users to search thousands of sources in parallel, requiring heavy duty machine translation capabilities. This will allow scientific discovery to rapidly accelerate.