A wide-field, high-resolution 3D exoscope for improved microsurgery workflow

Significance: Digital microscopes are becoming an indispensable tool during microsurgical operations. As opposed to their analog microscope counterparts, digital instruments now allow surgeons to view high-resolution images on heads-up 3D displays for improved ergonomics and team interaction. Digital microscopes also enable multiple clinicians to simultaneously monitor the surgical area of interest, record video data and facilitate annotation, and can connect with robotic surgery tools, for example. Unfortunately, due to optical lens and sensor design limitations, current surgical microscopes (either analog or digital) can only image a small area (andlt;1cm2) at high resolution. This limited area effectively forces the surgical team to view their operating area through a soda straw. One of the primary drawbacks is that the surgeon must repeatedly reposition and refocus the microscope head to bring new areas of interest into view, which can take a significant amount of time away from completing important tasks and also cause surgeons to lose their area of focus at crucial times, leading to longer surgeries and increased risk of complication. Proposal: Ramona Optics is developing a new surgical microscope system, termed the “micro-camera array exoscope” (M- CARE), which can overcome the limited viewing area of current instruments, to simultaneously capture 3D video data from a macroscopic area (7x10 cm) at extremely fine resolution (down to 4 µm/pixel). This results in the ability to have an instrument that does not require any physical re-positioning for nearly all surgical operations. Instead, the surgery team can digitally pan and zoom to different areas of interest to display up to 4 unique 2.5 cm2 viewing areas of the surgical space on up to different 4K 3D monitors positioned around the surgical area. We are hopeful that these new capabilities will lead to significantly faster micro- surgical operations with fewer complications. At the same time, our new instrument will allow surgical teams to independently monitor different surgical areas at high-resolution for improved workflow. Work executed in this Phase I proposal will set the stage for a Phase II effort that aims to first create a prototype instrument that digitally tracks relevant surgical regions for a completely hands-free imaging device and then test this instrument within the clinic.SA1: Lag-free 3D display at high-resolution across entire surgical area: Ramona Optics will complete development of a prototype M-CARE instrument that offers a 7x10 cm total 3D stereoscopic field-of-view (FOV) at 4 µm/pixel resolution with andlt;40 ms per frame latency. Users will have the ability to dynamically toggle between the imaging areas provided by 54 micro- cameras, which will be shown on a 4K autostereoscopic display for sight-line 3D viewing.SA2: Software for simultaneous multi-area imaging and display: We will create the necessary software to read up to 4 video streams from 4 unique FOVs for simultaneous viewing by different members of the surgery team.SA3: Seamless FOV tracking and 3D imaging quality assessment: The final goal of this Phase I proposal is to achieve on-FPGA image stitching within latency requirements (andlt;40 ms) to allow users to seamlessly move the desired FOV to arbitrary locations across the surgical area. SA3 will conclude with a series of user trials with partner surgeons, who will test the system on phantom tissue targets and provide valuable feedback on the hardware and software interface for subsequent Phase II development.Current digital surgical microscopes can only see a small andlt; 1 cm2 area when used to image at high- resolution (andlt;10 µm/pixel) and thus must be frequently repositioned to allow surgeons to effectively see entire operating regions. In this project, Ramona Optics will develop and test a new micro-camera array exoscope (M-CARE) to simultaneously image in 3D across a large 7x10 cm area at high resolution (andlt;10 µm) for heads-up display. A seamless digital interface will allow the surgical team to independently navigate across the surgical area without the need for any microscope re-positioning or re-focusing, which will lead to reduced operating times and risk of complication.