Deep Learning Architecture for a Wide Variety of Sensors

A multi-sensor fusion capability is proposed. Using a form of deep learning, inputs from multiple sensors are fused, interpreted, and output. Memory within the scene is preserved and used to better identify and track moving targets. Uncertainty is explicitly measured and output. Deep learning can represent arbitrary input data as continuous vector embeddings. These encapsulate semantics of input sources such as radar, lidar, and cameras. They have the unique ability to be combined into a single embedding that represents all input information. Thus, deep learning is an ideal candidate technology for sensor fusion. However, a valid criticism of remains. Neural networks are difficult to interpret. Our solution overcomes this critical challenge, lending much more interpretability and intuitive understanding. Additionally, our solution is specifically tailored to radar and lidar in a way that preserves native fidelity rather than discretizing to a raster format. In this phase, we build on existing work we have performed in regards to machine learning for radar, sensor fusion, and deep learning solutions to formulate the novel architecture. Demonstrations are performed using a state-of-the-art simulator capable of rendering sensor outputs and the required environmental conditions. Feasibility is determined via trial runs through urban environments