A Multidimensional Photonic Accelerator for Matrix Multiplications

We propose a photonic matrix accelerator (PMA) using coherent mixing and multi-dimensional detection performing matrix multiplications for high-efficiency neural network processing. Recent progress in artificial intelligence (AI) can be greatly attributed to the development in deep artificial neural networks (ANNs). Though ANNs are dominantly trained and deployed with digital computers, biological neural networks are solely based on analog systems with randomness and limited precision. The energy efficiency of biological neural networks is far superior to that implemented with digital ICs so far. For large scale information transmission and processing, optics has intrinsic advantages over electronics, thanks to its broad bandwidth and supporting large-scale mode multiplexing.  In this proposal we focus on photonic implementation of matrix multiplication, one of the fundamental yet most computational demanding calculations in ANN. In digital computer, matrix multiplication is carried out though large amount of multiply-accumulate (MAC) operations, which has large energy expenditure on memory access. The matrix multiplication in the proposed matrix accelerator does not require multiple register read/write of partial summation results, greatly reducing the energy consumption in memory accesses. The proposed matrix accelerator does not perform bit-level logical operation, but an analog computing device based on coherent detection dedicated to accelerating the MAC operations with unprecedented scalability and low power dissipation. Specifically, in this proposal we will Build a matrix accelerator prototype based on free space coherent mixing in spatial mode, which can perform 4x4 matrix-vector multiplication in one clock cycle. Demonstrate a matrix-matrix multiplier with polarization, wavelength parallelization. With a modulation/detection frequency at 32 GHz, the PMA will achieve a peak performance of 4.1 tera operations per second (TOPS), with a100 MHz matrix loading rate, and a total power consumption 260 TOPS with a device size of ~ 10 mm2, a 1.6 GHz matrix loading rate, and a power consumption < 10 W. Our team is composed of members with extensive expertise in photonics devices, optical communication, machine learning, and rich experience in technology transfer and commercialization.