Machine Learning for Distributed Traffic Control

Traditional systems for traffic management attempt to influence vehicle flow by controlling traffic signals or ramp meters. However, these approaches ignore an important component of traffic management: coordination of the vehicles themselves.

In this project, we explored a novel approach to traffic management that relies on a distributed scheme in which vehicles themselves select lanes, speeds, and routes. A simulated highway network populated by thousands of vehicles provides the research testbed. A reinforcement learning system acquires control strategies based on experience with this simulator, and a separate module learns to predict the expected loads along alternative routes. The reward function involves not individual vehicles but overall traffic behavior, as measured by the average squared difference between vehicles' actual and desired speeds.

Experimental studies have demonstrated that the strategies learned by this system let drivers more closely match their desired speeds than do handcrafted strategies, while also reducing the number of lane changes. They also suggest that the learned behaviors generalize to different traffic densities, different numbers of lanes, situations involving blocked lanes, and even the presence of selfish drivers.

This work was supported by (and carried out at) the DaimlerChrysler Research and Technology Center. Researchers who contributed to the project included David Moriarty , Pat Langley, Simon Handley, and Mark Pendrith.

Related Papers