Web-Based Scientific Collaboratory for Large Data Analysis

Simulation and experimental data is growing in size and complexity. As a result it is difficult to understand and leverage the value of these data. Thus new methods of analyzing and processing data must be found if we are to benefit from the wealth of data available now, not to mention the flood of data coming in the future. We are proposing to develop scientific collaboraties combining existing high-performance computing tools and social Web-collaboration tools, plus scalable computing technologies including data management and provenance. In addition, we will exploit the notion of VisMashUps, which are combinations of workflows and data created in an interactive, collaborative environment by the scientific community, and deployed and exchanged through Web-based interfaces. Kitware has helped develop several important visualization tools used in the DOE labs including VTK, ParaView and VisIt. We are teaming with VisTrails, Inc. a small firm supporting and developing the VisTrails application, which is widely used in DOE and known for its data provenance technology. More recently, VisTrails has introduced the notion of visualization mashups, enabling users to create, modify and exchange data processing workflows. In Phase I we will combine these applications, and create a prototype scientific collaboratory environment. If successful, in Phase II, we expect to extend the proposed framework to streamline the creation and exchange of VisMashUps across the community, and to simplify the installation and configuration of scientific collaboratories. Commercial Applications and Other Benefits The simulation and analysis market is large measured in the billions of dollars. Visual analysis tools such as those proposed will enable these techniques to be used more effectively, therefore the market potential for this technology is measured in the hundreds of millions of dollars. As an open source company, Kitware will contribute to the public good, including DOE, by delivering the core technology in open source form. However, we will commercialize the technology by providing workflow integration services to our customers and collaborators.