Optoelectronic Synapse-based Physical Reservoir Imager (OSPRI)

For unmanned aerial vehicles (UAVs) operating in GPS-deprived environments, visual markers are used for localization. Moreover, the UAV must likely perform all of the machine vision tasks onboard, without relying on cloud computing resources. There are two major limitations with this. One problem is that machine vision processing tasks require large amounts of energy. The second problem is that machine vision processing tasks also require large computational resources. This second problem can partially be reduced by using lite versions of machine vision software, but this can reduce the accuracy of the classification and also reduce the number of visual markers that can be classified. Physical reservoir computers (PRCs) are a type of non-conventional computing architecture. PRCs typically have several advantages, such as parallel processing, energy efficiency, and avoidance of the von Neumann bottleneck. Physical reservoir computing effectively repurposes a physical system to be utilized as a physical embodiment of a neural network “reservoir”. The reservoir, in the context of neural networks, is a section of nodes whose weights are left untrained. Optoelectronic synapses can act as a physical reservoir computer, in which the sensor can both store memory and compute some types of tasks. This provides an opportunity to embed machine learning capabilities at the hardware level of optoelectronic sensors. UAVs have limited energy, payload, and computing capacities. An Optoelectronic Synapse-based Physical Reservoir Imager (OSPRI) would decrease the required energy and CPU processing time necessary for machine vision, without adding additional weight. More than this, performing a significant portion of the computing on the optoelectronic sensor would also reduce the latency, which would allow faster response times. This on-sensor or near-sensor approach would allow UAVs operating GPS- and cloud-deprived environments to instead rely on visual markers. An optoelectronic synapse-based PRC would provide fast processing of visual markers, while also decreasing the energy and CPU burden of this task.