Metasurface Optical Neural Processor for Image Recognition

Object recognition has been exploited in a wide range of applications, including image annotation, vehicle counting and tracking, pedestrian detection, and facial detection and recognition. Technologies combining image processing and computer vision extract high-dimensional data from the real world and utilize machine learning algorithms to transform the data into lower dimensional representations that aid in understanding and interpretation of the images. The full technology stack requires a compound optical system to form images, an optoelectronic sensor for analog-to-digital conversion, and digital processors to implement artificial neural networks (ANNs). This process consumes at least milliwatts of power and is slow due to the latency between modules. It is also vulnerable to cyber-attack. The proposed project will investigate a new scheme to perform artificial neural computing based on the physics of wave dynamics. Specifically, we will demonstrate an optical neural processor (ONP) in the form of a smart glass that can recognize objects placed in front of it. The smart glass receives light scattered by the object and directly processes the optical information using its internal nanostructures. Our smart glass will be based on metasurfaces, which are composed of a two-dimensional array of meta-units and can offer complete and precise manipulation of optical amplitude and phase across the wavefront with subwavelength resolution. The collective operation of millions of meta-units with subwavelength dimensions enables efficient parallel computing with a high expressive power; as such, tasks typically solved using a complex multi-layered digital neural network can be accomplished by our ONP using just a single metasurface or a few cascaded metasurfaces. The proposed ONP does not need any power supply or digital processor: it acts as a passive computer that operates at the speed of light. Moreover, the physics-based computing that relies on intrinsic and engineered material properties offers security beyond digital encryption. On the foundation of our preliminary experimental work on recognition of handwritten digits and letters using metasurfaces, this project will design and implement ONPs to recognize a large number of classes of monochrome and color images illuminated by both coherent and incoherent light. To realize this goal, we will focus on increasing the width and depth (and thus the expressive power) of our optical neural network (e.g., by using polarization multiplexing and wavelength multiplexing in each metasurface, by using arrays of metasurfaces on each layer of the network, and by cascading metasurface layers). The success of the project can serve as a milestone in demonstrating the exciting potential of this emerging computing paradigm. The performance metrics such as energy consumption and latency are expected to exceed those of conventional digital ANNs by many orders of magnitude.