Robotic Following using Deep Learning

PhysicsAI proposes to develop, integrate, and demonstrate a prototype embedded system that can incrementally learn robotic following behaviors on a large vehicle, using deep learning algorithms and inexpensive COTS sensors including video cameras and radar. To scale up to large vehicles, the Phase II prototype will build on several key deep neural network (DNN) approaches that proved effective in Phase I simulations and laboratory results: 1) End-to-end deep learning for producing driving outputs directly from sensor inputs without using intermediate pipelined modules 2) Multi-task learning for accurately and efficiently producing multiple driving outputs from the same DNN model including leader position, path curvature, and speed 3) Spatio-temporal deep learning for providing more powerful machine perception of kinematics using memory of previous video frames 4) Correction-based incremental learning (CBIL) for incrementally improving with each run as the robotic follower learns from its mistakes during training runs. In Phase II, PhysicsAI will develop a prototype DNN-Kit including the embedded system design, interface control documents, software, and hardware. The DNN-Kit will be integrated onboard a large vehicle provided by the government, and demonstrated in real-world environments and operating conditions. We anticipate the Phase II effort will increase the TRL from 4 to 7.