Open Source Parallel Image Analysis and Machine Learning Pipeline

Continuum Analytics proposes a Python-based open-source data analysis machine learning pipeline toolkit for satellite data processing, weather and climate data processing, and machine learning and prediction with optional proprietary cluster management tools for streamlined deployment for cloud providers and on-premises clusters. The innovative software will empower scientists and analysts to readily and seamlessly construct and test workflows that transparently and scalably perform calculations across cluster nodes for data-driven discovery. The simple API for homogenous processing of images, mosaics and tiles further improves ease of use for rapid testing and prototyping of analyses paradigms for multiple extremely large data sets. Today, NASA researchers must create, debug, and tune custom workflows for each analysis. Creation and modification of custom workflows is fragile, non-portable, and consumes time that could be better spent on advancing scientific discovery. The Phase I work plan will demonstrate that it is feasible to easily create and compose data manipulations and analytics from a variety of sources with a portable, reproducible, extensible process that can be deployed on a wide variety of systems and software. This is a major improvement over the current state-of-the-art because of reduced workflow creation time, portability of deployment and use, extensibility, and robustness.