High Performance Computing (HPC) Tools for Topology Aware Mapping of Inter-node communication

A generalized network topology-aware algorithm has been developed that efficiently maps structured and unstructured domains onto available nodes to minimize MPI message passing costs. The methodology utilizes sub-domain inter-dependency information, readily available from standard domain decomposition codes, along with a simulated annealing optimization technique to automatically assign sub-blocks to tasks on available hardware nodes. Testing during Phase I highlighted runtime performance benefits using topology mapping and results suggest further runtime acceleration if the optimization procedure is extended to incorporate the domain partitioning process and OpenMP thread distribution. Message pass timing predictions will be enhanced using database-driven response surface procedure that can more accurately represent cost implications from heavy network traffic in a timely manner for supporting real-time domain mapping decisions. Improved overlapping of computational effort with data communication within a legacy code framework will also be undertaken. Required solver software modifications to utilize topology-aware zone placement results are straightforward to implement. As a performance benchmarking tool, the procedure can also guide software modifications needed to further reduce identified communication overhead costs and improve code strong scaling capability on current and emergent massively parallel computer architectures.