Rapid Multi-Modal Microscopy Feature Correlation Tool (RM-MFCT)

Nuclear forensic analysis is critical to preventing nuclear terrorism. This analysis involves combining information from multiple microanalytical imaging technologies and other spectroscopic methods in a serial fashion to provide all required information; specifically the size, morphology, composition, and molecular and elemental makeup of the samples. Image registration is a crucial step in all image analysis tasks in which the final information is gained from the combination of various data sources. Image registration can overcome issues such as image rotation, scale, and skew effect that are common when combining this data. However, this methodology has not been widely used to correlate features between optical microscopy platforms and analytical image platforms due to several challenges, including the large differences in the Region of Interest (ROI), resolution, and morphology. ChemImage Sensor Systems (CISS) proposes the development of the Rapid Multi-Modal Microscopy Feature Correlation Toolkit (RM-MFCT). RM-MFCT is a set of algorithms that provides rapid automated feature correlation between optical microscope images and images obtained by SEM, SEM/EDS, SIMS, and other analytical imaging methods. RM-MFCT combines the advantages of deep learning and computer vision, therefore, it can better address the key challenges during the feature correlation process, include large scale and morphology differences, blurred boundary, lack of salient features and skew effects. RM-MFCT provides the most viable path to a fully automated, cost-effective, small footprint design that provides rapid, robust and highly accurate feature correlation results. CISS will develop the RM-MFCT utilizing both the advanced deep learning technique and classical computer vision approach. The deep learning technique will be applied initially to quickly correct large scale and appearance differences between optical and analytical microscopic data. Then, we will fine-tune the registration results using the traditional computer vision methods.RM-MFCT would have applicability beyond the nuclear forensics industry. The automation of image registration has significant commercialization potential in federal and civilian markets, in any field that utilizes sensor fusion technique, particularly within the medical/surgical fields and security industries.