A Scalable Platform for Real-Time Microscopy Image Analysis Using Artificial Intelligence and Machine Learning

Quantitative digital microscopy involves identifying features within captured images and counting, categorizing, and measuring each feature to extract higher order metrics to make informed decisions in a variety of fields, such as materials, biology, metrology, medicine, and additive or advanced manufacturing. The quantitative digital microscopy workflow has been a low throughput affair with a high degree of skilled human labor that is difficult to scale in the current state to meet the demands of next generation microscopy and materials characterization systems. This is exacerbated by difficult to use legacy and often “quirky” data analysis software that cannot process multiple data streams in parallel or automatically adapt to variations in workflow and analysis requirements. The feasibility of an automated quantitative microscopy image analysis platform that leverages the latest techniques in artificial intelligence and machine learning and overcomes the issues of scale and adaptability has been demonstrated. The proposed effort then seeks to refine this technology for full-scale commercialization. The technology, which uses a collection of independent graphical processing unit-equipped edge computing devices running state-of-the-art artificial intelligence/machine learning models for simultaneous automated image analysis, was demonstrated at a national user facility with a multi-detector electron microscope for materials characterization. This technology will be further advanced to deploy on most microscopy instruments in a user facility while still maintaining an easy-to-use software interface and seamless integration with modern microscopy workflows. The proposed effort will then extend past the single experiment and use case demonstrated in the initial feasibility study by leveraging the zero-configuration and platform agnostic nature of the technology. Specifically, the technology’s edge computing device and user interface stacks will be advanced through concentrated efforts on optimizing network communication, analysis pipelines, and visualizations for near-infinite device growth and extensibility to enable widespread adoption in industry, energy, and medical sectors using optical, charged particle, and scanning probe microscopes. The resulting outcome will be a scalable, future-proof product that ultimately converts a low throughput, manual data analysis process into a high throughput, automated platform that accelerates the innovation lifecycle of new technologies that rely on quantitative microscopy with multiple data streams, such as products and processes under development in materials science, biology, medical imaging, geology, energy storage, and energy consumption.