Multiscale Fast and Distributed Data and Statistics Summarization

Over the last decade the amount of data available from the internet, sensors and other sources has grown dramatically providing the opportunity to gain novel insights in many fields. Uncovering non-linear low dimensional structure in high-dimensional data (i.e., manifold learning), a key to summarization, remains a challenging problem which ultimately inhibits knowledge discovery. Intelligent Automation Inc, in collaboration with Johns Hopkins University proposes the developed and application of scalable and robust multiresolution summarization algorithms to several relevant problems. The methods proposed are based on Geometric Multi-Resolution Analysis and Diffusion Geometries which enjoy provable accuracy and scaling guarantees. These approaches have many favorable characteristics including linear scaling with the number of data points, multi-resolution representation of the data, robust to noise, provable error estimates, amenable to fast algorithms, and suitability for visualization and subsequent analysis. We will develop parallel and distributed implementations to support execution of summarization method on large amounts of high dimensional in data intensive computing environments. To demonstrate the benefits of these methods will we develop a visualization capability that enable interactive knowledge discovery and apply them to problems in text and hyperspectral image analysis.