A Data Science and Machine Learning Platform Supporting Large Particle Accelerator Control and Diagnostics Applications

We propose an infrastructure to collect, transport, store, manage, and retrieve data as a general platform for the development of data science and machine learning applications necessary for the control and diagnosis of large particle accelerator facilities, and associated beamline instrumentation. We aim to acquire data at sampling capacities required for real-time (i.e., online) control applications, the proposed objective being an acquisition rate of 1 kHz for up to 4,000 data signals. The novel aspect of the proposed platform is immediate and comprehensive data accessibility. Specific features include the ability to access, search, and process data as it is collected and stored in real time and extensive search and analysis capabilities of archived data necessary to support general data science and machine learning applications. This functionality is achieved using a self-describing data storage format and archive manager capable of maintaining full data provenance, user-defined classifications, annotations, artifacts, and data correlations This data store also has a well-defined API (Application Programming Interface) for advanced post-acquisition analysis. The objective is to develop a platform that minimizes facility independence and generalizes the training and deployment of machine learning algorithms for different operating configurations for the same beamline, or between facilities. The platform can be deployed at any facility utilizing the EPICS control system. In Phase I, the prototype system is built in-house using simulated data sources. The Phase II effort entails deploying the data framework at different accelerator facilities utilizing EPICS, prime candidates being the High-repetition Rate Electron Scattering (HiRES) apparatus accelerator facility at the Lawrence Berkeley National Laboratory at the direction of Daniele Filippetto and Carlos Serrano, and the Stanford Linear Accelerator (SLAC) facility in Palo Alto under the supervision of Greg White. Phase II also contains the development of enhanced data throughput, advanced data analysis features, and the development of dedicated machine learning algorithms as plugins. B Project Summary The Phase I effort can be subdivided into a set of research and development tasks, and performance studies, necessary for the construction of the data platform. These tasks are identified as follows: 1) Develop a simulation data source to emulate fast FPGA data sources. 2) Develop aggregation/compression library to merge data into tables and multi-dimensional arrays. 3) Develop a file writer client that creates a self-describing archive, complete with indexing, management capabilities, and full provenance. 4) Develop an archive manager client supporting machine learning and data mining capabilities. 5) Performance evaluation: characterize throughput of all data handling parts of the architecture. Phase II contains, but is not limited to, the following tasks: 1) Installation, testing, and continued development at HiRES, a compact accelerator facility. 2) Installation and testing at SLAC, a large accelerator facility. 3) Development of advanced data compression and aggregation algorithms and techniques. 4) Continued development of data analysis and data mining features during deployment. 5) Development of fast, dedicated machine learning algorithms to be deployed as platform plugins. Further evaluations will continue post deployment.