Intelligent Urban Traffic Flow Optimization Using Multi-Agent Deep Q-Learning and Spatial-Temporal Convolutional Networks

Abstract

Traffic congestion in urban areas remains one of the biggest issues for the modern city. Traffic congestion causes
longer travel time, increased fuel consumption and increased emissions. This paper presents an intelligent traffic
management system which uses Multi-Agent Deep Q-Learning (MADQL) and Spatial-Temporal Convolutional
Networks (STCN) to provide real-time management of traffic signals. using MADQL, we model our urban
intersections as a multi-agent environment where each traffic light is considered an agent acting independently in
order to learn the optimal control policy over time using multi-agent harmonic-reinforcement learning. The STCN
captures complex spatial-temporal traffic patterns across the urban network which provides state representations
that are significantly rich in information for our actions. We validated the efficacy of the system in real-life length
scenarios with a reduction in average waiting time of at least 34%, reduction in fuel consumption of at least 28%
and at least 25% increase in intersection throughput. Our system also has a distributed architecture allowing it to
scale easily in larger urban networks while satisfying the real-time requirements when traffic signals establish
new states.