Preparing Raw Data for Consumption by AI/ML Services

We are developing a novel Extract-Transform-Load (ETL) platform to accelerate the preprocessing of raw unstructured data (i.e. natural language, image, and audio) into data that not only is discoverable but also ready for consumption downstream AI/ML services (e.g. data annotation, model training, production pipelines). Currently, data scientists spend more than 80% of their time on data engineering; our goal is to give this time back to them by reducing the amount of data engineering required on novel unstructured data types by 90%. Initially, our suite of open source (and on-prem) tooling will enable data scientists and data engineers to rapidly create preprocessing engines to transform raw natural language data (e.g. PDF, XML, HTML, etc.) into JSON and other relevant formats. These output files will already be mapped to the required data schema of the downstream data lake or AI/ML service, making the output files immediately ready consumption without additional data engineering work.