Reward Scheme Analysis for DQN-Based Adaptive Traffic Signal Control in Road Repair Zones

Authors

  • Yoanda Alim Syahbana Politeknik Caltex Riau
  • Muhammad Wahyudi Politeknik Caltex Riau
  • Fikri Muhaffizh Imani Politeknik Caltex Riau
  • Ardianto Wibowo IMT Atlantique Brest
  • Fatur Rizky Al-Farisz Politeknik Caltex Riau

DOI:

https://doi.org/10.29207/resti.v10i3.7381

Keywords:

adaptive traffic signals, deep Q-network, reinforcement learning, reward schemes, SUMO simulation

Abstract

Traffic congestion caused by temporary road repairs often forces bidirectional traffic to alternate through a single lane, leading to increased delays and imbalanced traffic flow. This study investigates the impact of reward scheme design on the performance of a Deep Q-Network (DQN)-based adaptive traffic signal control system in such constrained environments. Using the Simulation of Urban Mobility (SUMO), a traffic scenario involving 1,656 vehicles over 1,800 seconds was modeled to evaluate six reward scheme configurations combining Traffic Flow (TF), Waiting Time (WT), and Average Speed (AS): TF-TF, TF-WT, WT-TF, WT-WT, AS-TF, and AS-WT. The DQN agent, implemented with a two-layer neural network and trained for 50 epochs, dynamically adjusted signal timing to balance traffic from opposing directions. Experimental results indicate that the AS-WT configuration achieved the most balanced performance, producing the best fairness index (1.04) while maintaining stable traffic flow. In contrast, schemes with misaligned or redundant metrics showed significantly poorer performance. These findings highlight the importance of reward design in reinforcement learning-based traffic signal control and suggest that carefully selected reward schemes can improve fairness and efficiency in temporary road repair zones.

Downloads

Download data is not yet available.

References

H. B. Faheem, A. M. El Shorbagy, and M. E. Gabr, "Impact of Traffic Congestion on Transportation System: Challenges and Remediations - A Review," Mansoura Eng. J., vol. 49, no. 2, pp. 1–15, Jan. 2024, doi: 10.58491/2735-4202.3191.

T. Afrin and N. Yodo, "A Survey of Road Traffic Congestion Measures Towards a Sustainable and Resilient Transportation System," Sustainability, vol. 12, no. 11, p. 4660, Jun. 2020, doi: 10.3390/su12114660.

B. Wang, Z. He, J. Sheng, and Y. Chen, "Deep Reinforcement Learning for Traffic Light Timing Optimization," Processes, vol. 10, no. 11, p. 2458, Nov. 2022, doi: 10.3390/pr10112458.

L. Koch, T. Brinkmann, M. Wegener, K. Badalian, and J. Andert, "Adaptive Traffic Light Control with Deep Reinforcement Learning: An Evaluation of Traffic Flow and Energy Consumption," IEEE Trans. Intell. Transp. Syst., vol. 24, no. 12, pp. 15066–15076, Dec. 2023, doi: 10.1109/TITS.2023.3305548.

R. Kumar, N. V. K. Sharma, and V. K. Chaurasiya, "Adaptive Traffic Light Control Using Deep Reinforcement Learning Technique," Multimed. Tools Appl., vol. 83, no. 5, pp. 13851–13872, Feb. 2024, doi: 10.1007/s11042-023-16112-3.

F. Rasheed, K.-L. A. Yau, R. M. Noor, C. Wu, and Y.-C. Low, "Deep Reinforcement Learning for Traffic Signal Control: A Review," IEEE Access, vol. 8, pp. 208016–208044, 2020, doi: 10.1109/ACCESS.2020.3034141.

W. Jia and M. Ji, "Multi-agent Deep Reinforcement Learning for Large-scale Traffic Signal Control with Spatio-temporal Attention Mechanism," Appl. Sci., vol. 15, no. 15, p. 8605, Aug. 2025, doi: 10.3390/app15158605.

M. Elharoun, S. M. El-Badawy, E. A.-E. Shwaly, and U. E. Shahdah, "Adaptive Traffic Signal Control Using Deep Reinforcement Learning: A Multi-objective Approach for Single and Multi-intersection Scenarios," IATSS Res., vol. 49, no. 4, pp. 481–492, Dec. 2025, doi: 10.1016/j.iatssr.2025.10.004.

A. Saadi, N. Abghour, and Z. Chiba, "A Survey of Reinforcement and Deep Reinforcement Learning for Coordination in Intelligent Traffic Light Control," J. Big Data, vol. 12, no. 1, p. 84, 2025, doi: 10.1186/s40537-025-01104-x.

P. Michailidis, I. Michailidis, C. R. Lazaridis, and E. Kosmatopoulos, "Traffic Signal Control via Reinforcement Learning: A Review on Applications and Innovations," Infrastructures, vol. 10, no. 5, p. 114, May 2025, doi: 10.3390/infrastructures10050114.

K. L. Tan, S. Poddar, A. Sharma, and S. Sarkar, "Deep Reinforcement Learning for Adaptive Traffic Signal Control," in Proc. IEEE 22nd Int. Conf. Intell. Transp. Syst. (ITSC), Auckland, NZ, 2019, pp. 353–358, doi: https://doi.org/10.48550/arXiv.1911.06294.

V. Mnih et al., "Human-level Control Through Deep Reinforcement Learning," Nature, vol. 518, no. 7540, pp. 529–533, Feb. 2015, doi: 10.1038/nature14236.

P. A. Lopez et al., "Microscopic Traffic Simulation Using SUMO," in Proc. 21st Int. Conf. Intell. Transp. Syst. (ITSC), Maui, HI, USA, 2018, pp. 2575–2582, doi: 10.1109/ITSC.2018.8569938.

Y. H. Chow, K. J. A. Ooi, M. A. S. Bhuiyan, M. B. I. Reaz, and C. W. Yuen, "Computation and Optimization of Traffic Network Topologies Using Eclipse SUMO," Ann. Emerg. Technol. Comput., vol. 6, no. 4, pp. 31–37, Oct. 2022, doi: 10.33166/AETIC.2022.04.004.

L. Kuyer, S. Whiteson, B. Bakker, and N. Vlassis, "Multiagent Reinforcement Learning for Urban Traffic Control Using Coordination Graphs," in Proc. Eur. Conf. Mach. Learn. Knowl. Discov. Databases (ECML PKDD), Antwerp, Belgium, 2008, pp. 656–671, doi: 10.1007/978-3-540-87479-9_61.

K.-L. A. Yau, J. Qadir, H. L. Khoo, M. H. Ling, and P. Komisarczuk, "A Survey on Reinforcement Learning Models and Algorithms for Traffic Signal Control," ACM Comput. Surv., vol. 50, no. 3, Art. no. 34, May 2018, doi: 10.1145/3068287.

H. Luo and S. Jin, "Reinforcement Learning for Traffic Signal Control in Hybrid Action Space," IEEE Trans. Intell. Transp. Syst., vol. 25, no. 6, pp. 5225–5241, Jun. 2024, doi: 10.1109/TITS.2023.3349248.

C. Li, X. Ma, L. Xia, Q. Zhao, and J. Yang, "Fairness Control of Traffic Light via Deep Reinforcement Learning," in Proc. IEEE 16th Int. Conf. Autom. Sci. Eng. (CASE), Hong Kong, 2020, pp. 752–757, doi: 10.1109/CASE48305.2020.9217032.

Eclipse Foundation, "SUMO User Documentation," SUMO - Simulation of Urban Mobility, 2025. [Online]. Available: https://sumo.dlr.de/docs/index.html. [Accessed: Mar. 20, 2025].

I. Essamlali, H. Nhaila, and M. E. Khaili, "Impact of Urban Block Shape on Traffic and Air Quality: A SUMO-based Comparative Study of Rectangular, Radial, and Triangular Forms," Transp. Res. Interdiscip. Perspect., vol. 31, p. 101413, 2025, doi: 10.1016/j.trip.2024.101413.

A. Sevim, Q. Guo, and E. E. Ozguven, "A Simulation-based Framework for Leveraging Shared Autonomous Vehicles to Enhance Disaster Evacuations in Rural Regions with a Focus on Vulnerable Populations," J. Infrastruct. Preserv. Resilience, vol. 6, no. 1, p. 10, 2025, doi: 10.1186/s43065-024-00093-y.

R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed. Cambridge, MA, USA: MIT Press, 2018.

C. Szepesvári, "Algorithms for Reinforcement Learning," Synth. Lect. Artif. Intell. Mach. Learn., vol. 4, no. 1, pp. 1–103, 2010, doi: 10.1007/978-3-031-01551-9.

D. Silver et al., "A General Reinforcement Learning Algorithm That Masters Chess, Shogi, and Go Through Self-play," Science, vol. 362, no. 6419, pp. 1140–1144, Dec. 2018, doi: 10.1126/science.aar6404.

M. Koziarski and B. Cyganek, "Impact of Low Resolution on Image Recognition with Deep Neural Networks: An Experimental Study," Int. J. Appl. Math. Comput. Sci., vol. 28, no. 4, pp. 735–744, 2018, doi: 10.2478/amcs-2018-0056.

M. Nazari, A. Oroojlooy, L. Snyder, and M. Takac, "Reinforcement Learning for Solving the Vehicle Routing Problem," in Proc. 32nd Int. Conf. Neural Inf. Process. Syst. (NeurIPS), Montréal, Canada, 2018, pp. 9861–9871, doi: https://doi.org/10.48550/arXiv.1802.04240.

S. Yoo, J. B. Lee, and H. Han, "A Reinforcement Learning Approach for Bus Network Design and Frequency Setting Optimisation," Public Transp., vol. 15, no. 2, pp. 503–534, Jun. 2023, doi: 10.1007/s12469-022-00319-y.

M. Roderick, J. MacGlashan, and S. Tellex, "Implementing the Deep Q-Network," arXiv, 2017, doi: https://doi.org/10.48550/arXiv.1711.07478.

S. Park, E. Han, S. Park, H. Jeong, and I. Yun, "Deep Q-network-based Traffic Signal Control Models," PLoS ONE, vol. 16, no. 9, p. e0256405, Sep. 2021, doi: 10.1371/journal.pone.0256405.

M. Dave, "Dynamic Traffic Light Management System," GitHub, 2021. [Online]. Available: https://github.com/Maunish-dave/Dynamic-Traffic-light-management-system. [Accessed: Mar. 20, 2025].

C. Cai and M. Wei, “Adaptive urban traffic signal control based on enhanced deep reinforcement learning,” Scientific Reports, vol. 14, no. 1, Art. no. 14116, Jun. 2024, doi: 10.1038/s41598-024-64885-w.

K. Cao, L. Wang, S. Zhang, L. Duan, G. Jiang, S. Sfarra, H. Zhang, and H. Jung, “Optimization Control of Adaptive Traffic Signal with Deep Reinforcement Learning,” Electronics, vol. 13, no. 1, p. 198, Jan. 2024, doi: 10.3390/electronics13010198.

Y. Zheng, J. Luo, H. Gao, Y. Zhou, and K. Li, “Pri-DDQN: learning adaptive traffic signal control strategy through a hybrid agent,” Complex & Intelligent Systems, vol. 11, Art. no. 47, 2025, doi: 10.1007/s40747-024-01651-5.

J. Bao, C. Wu, Y. Lin, L. Zhong, X. Chen, and R. Yin, “A scalable approach to optimize traffic signal control with federated reinforcement learning,” Scientific Reports, vol. 13, Art. no. 19184, Nov. 2023, doi: 10.1038/s41598-023-46074-3

Downloads

Published

2026-06-08

Issue

Section

Artificial Intelligence

How to Cite

Syahbana, Y. A., Wahyudi, M. ., Imani, F. M., Wibowo, A., & Al-Farisz, F. R. (2026). Reward Scheme Analysis for DQN-Based Adaptive Traffic Signal Control in Road Repair Zones. Jurnal RESTI (Rekayasa Sistem Dan Teknologi Informasi), 10(3), 609-616. https://doi.org/10.29207/resti.v10i3.7381