A Python Interface to Trilinos/Tpetra for High-Level Access to HPC Solvers

The need to access parallel computing resources is ever expanding, as both the scale of simulations and the size of data sets are quickly growing. Also, the increasing ubiquity of parallel hardwaremulti-core chips, parallel co-processors and accelerators, near effortless cluster computing on cloud infrastructurespush new and existing high-performance computing (HPC) users to move to parallel programming. Unfortunately, the gap is increasing between the available performance offered by HPC systems and the available usability of the software packages used to program them. We propose to develop a distributed HPC framework composed of three open-source software packages in the user-friendly and expressive Python language. This framework builds on the success of Phase I by (i) continuing the development of improved PyTrilinos wrappers, (ii) by implementing a general distributed array package modeled after NumPy and, (iii) by providing functionality to easily import external routines into Python, including parallel code. This Python HPC framework will make parallel computing easier to use by bringing the user-friendly NumPy interface to HPC, more intuitive by reducing the mental overhead required to work with parallel distributed arrays, and more useful by enabling access to parallel distributed solvers. PyTrilinos wrappers were expanded to wrap fundamental components of the second-generation Tpetra package, providing a basis for further expansion of PyTrilinos wrappers. The second Phase I success was that the Optimized Distributed NumPy package was designed and deemed feasible from both a technical and business perspective. A distributed HPC framework will be created to provide commonality for parallel programming in Python and allow easy access to external HPC libraries. This includes building on the research and development of Phase I by continuing the PyTrilinos expansion and implementing the distributed array package designed in Phase I, as well as a third package that easily integrates external code into Python and compiles specific routines. Commercial Applications and Other Benefits: Enthought will use the framework to (i) enhance its software product offerings, (ii) to provide its consulting clients with large simulation and modeling based consulting and data analysis and, (iii) to provide HPC-with-Python software training. In addition, the distributed Python HPC framework will be made freely available as open-source software, providing a basis for others on which to build commercial applications and with which to solve complex problems.