CoRTES - Continuous Robotics AI/ML Training, Evaluation & Deployment System

UGVs are finding numerous applications in both civilian and defense use cases. In the military, UGVs can be configured for different roles, including reconnaissance, target acquisition, explosive ordnance disposal, communications relay, firefighting, medical evacuation, etc. Considering these high-value applications, many organizations such as DEVCOM GVSC, ARL, and DARPA are working towards improving UGV autonomy using artificial intelligence (AI) or Deep Learning based models for perception, planning, and control. Through empirical studies, it has been shown that DL models trained on limited data are capable of generalizing well to new unseen data. This makes them highly suitable for military-related applications where the robots could be deployed in unknown and unstructured environments for which labeled datasets may not be readily available. However, their generalization capabilities are bounded. If the newly encountered scenarios are vastly different from the training data, DL models can struggle to deliver the desired performance. To mitigate these limitations, learning-based models are improved by iteratively training and retraining them, as the UGVs collect new data from field experiments. Presently, this entire process which involves data collection, labeling, organizing, training, testing, and deployment on UGVs is highly manual, cumbersome, and time-consuming, causing substantial delays in autonomy-related research. There exists no tool that automates and streamlines this process.Ā In this phase II proposal, Inception Robotics proposes to develop CoRTES - Continuous Robotics AI/ML Training, Evaluation & Deployment System to streamline the entire process of training DL models, as UGVs collect more data from real-world deployments. The primary objective of the Phase II project is to develop a robust prototype of the training pipeline, with a Technology Readiness Level of 6. CoRTES simplifies how raw sensor data from UGVs, logs, system metrics, etc. are collected, visualized for researchers, and stored (locally, and in the cloud) with automatically generated metadata for quick future retrieval. Our system also integrates manual data annotation tools enabling researchers to create labels for the newly collected data, and modules for DL model training. CoRTES employs a modular design that can accommodate any simulator such as Isaac Sim, Gazebo, Unity, etc. for testing the performance of the trained models. It automatically records the evaluations, visualizes metrics, and if a model surpasses user-defined performance metrics during testing, automatically deploys it to a UGV for field experiments. Our prototype can operate on the local (on-premise) server setup used by Army researchers to ensure high data security and does not depend on the availability of internet connectivity. From our Phase I project, we have estimated a time saving of at least 50% throughout the DL model training process.