Offline Reinforcement Learning-Based Traffic Management to Improve System Wide Energy Efficiency

Qen Labs Inc. is pioneering the development of a citywide traffic management system that adeptly navigates the multifaceted challenges associated with treating automatic traffic signal control as a multi-objective optimization problem. Distinctively, our methodology circumvents the need for real-world traffic system trials by harnessing Offline Reinforcement Learning (Offline-RL). This technique necessitates detailed vehicular movement data to refine traffic control strategies. Our holistic strategy incorporates an amalgamation of historical data, taxi trip records, crowdsourced trajectories, generative AI-created trajectories, and simulated data to construct a rich dataset for training our offline-RL model, which yields traffic signal control policies. By leveraging a diverse array of data sources, we obtain a multi-layered perspective on urban mobility patterns, vital for the planning, management, and optimization of transportation networks. Conventional simulator-based traffic modeling, which falls short in integrating historical traffic data, is eclipsed by our approach. Moreover, the use of real-world reinforcement learning experiments, fraught with ethical concerns due to the imposition of untested systems on actual traffic flows, is replaced by our novel application of offline reinforcement learning. This approach utilizes real-world traffic data to optimize traffic management objectives, notably enhancing system-level energy efficiency without the risk of operational disruptions. Our use of Offline-RL has demonstrated up to a 30% improvement in traffic flows at signal intersections, marking a significant advancement in traffic management solutions. Finally, we will make edge level implementations possible to ensure swift decision-making and significantly reduce latency, thereby enhancing the efficiency and responsiveness of the urban traffic management ecosystem.