Topological Robust Algorithms for Massive Data Sets via Agent-based Modular Infrastructure (TA-DA) Supporting Decentralized and Parallel Processing

We propose to develop topological robust dimensionality reduction algorithms for massive data sets. The algorithm will be wrapped in an agent-based modular infrastructure and will support decentralized and parallel data processing. Our algorithms are based on Mapper, a method that discovers the data topology by constructing lower-dimensional simplicial complexes. Our goal is to reduce the role of the mathematician in selecting appropriate topological and geometric parameters of Mapper, and automate the process in a parallel fashion. The parallelization is already inherent to Mapper, due to its partial clustering. We will improve the parallelization by borrowing ideas from streaming algorithms and construction of local wise alpha-complexes. Robustness of the data discovery process to noisy and missing data is built on persistent homology, where only persistent structures remain and others collapse. Automatic assessment of the algorithm will be handled by comparing multiresolutional outputs, for example, real vs. artifact features. Querying of data will be handled by topological matching in the constructed simplicial complexes. The algorithmic infrastructure will be wrapped by a family of software agents using Intelligent Automation’s agent infrastructure Cybele. As part of the infrastructure development, APIs will be developed to assist in the modularization.