Enabling Next Generation Machine Learning for Large Scale Image Analysis

Project Summary/Abstract Deep learning has transformed medical image analysis by delivering clinically meaningful results on challenging problems like prostate cancer detection and lung screening. In pathology, industry is making significant invest- ments to develop deep learning tools for diagnostic use in clinical labs. FDA approval of whole-slide digital pathology images (WSIs) for use in primary diagnosis is further increasing interest, adoption, and investment in this technology. Judgments made by pathologists are the basis for the treatment of many diseases, yet in- terobserver variability among pathologists is significant, and errors can lead to overtreatment or even treatment of healthy patients. Pathology is also facing workforce issues as demand for pathologist services is outpacing growth of trained pathologists. Computational pathology tools based on deep learning can help address these problems by providing reproducible diagnoses, performing ”second reads” for human pathologists, automating tasks to improve pathologist efficiency, and helping general pathologists evaluate challenging cases. GPU accel- erators have played a significant role in advancing deep learning methods to build computational pathology tools, with machine learning frameworks (MLFs) like Pytorch and Tensorflow providing researchers with abstractions to quickly develop models that utilize GPUs. Evolution of GPUs and MLFs has been driven by analysis of small images, and so these tools cannot be easily applied directly WSIs or other large medical images like three dimen- sional MRI or CT. Adapting medical imaging problems to small image paradigms supported by GPUs and MLFs leads to suboptimal performance and increased implementation effort and complexity. More recent approaches that use streaming or ”unified memory” allow direct analysis of entire WSIs and have demonstrated performance advantages. These approaches can be slow, complex to implement, and are highly specific to a choice of network architecture which limits exploration and development of new architectures. More general-purpose, efficient, and user-friendly frameworks are required to allow the development of WSI scale deep learning.This project will develop techniques to automatically map deep learning networks implemented in common MLF architectures to one or more GPUs for arbitrarily large input images and activation layers. The proposed software will include a performance modeler to estimate the runtime of a given network on available GPU acceler- ators. These strategies will enable a new paradigm in deep learning for medical images, allowing the development of novel networks that are purpose-built for medical applications. Developers will be able to rapidly create and evaluate these networks using familiar MLF packages. This project will provide approaches to overcome GPU memory bottlenecks, a scheduler to map the network to available GPUs, integration with common MLFs, and demonstration using computational pathology use cases.Narrative The proposed tools will help software developers overcome the limitations of current computing hardware to design more accurate deep learning models for use in clinical diagnostics. These models will be able to analyze very large digitized images of glass slides to aid pathologists in tasks like cancer detection. The ability to analyze these images in their entirety instead of in small parts will improve the diagnostic accuracy of models and will accelerate algorithm development efforts.