Hybrid Deep Reinforcement Learning and Particle Swarm Optimization for Accelerated Multipath Routing in Congested SDN Environments

Authors

  • Rissal Efendi Universitas Kristen Satya Wacana

DOI:

https://doi.org/10.29207/resti.v10i2.7324

Keywords:

adaptive routing, Hybrid DRL-PSO, network congestion, optimization, performance evaluation

Abstract

Network congestion remains a critical challenge in dynamic communication environments, often degrading data delivery performance and Quality of Service (QoS). This study proposes a Hybrid Deep Reinforcement Learning with Particle Swarm Optimization (DRL–PSO) model to adaptively optimize routing paths and mitigate congestion in Software Defined Networking (SDN). The proposed approach integrates the exploration capability of Deep Reinforcement Learning with the fast convergence characteristics of Particle Swarm Optimization to select optimal routes based on real-time network conditions. Simulations were conducted using realistic network topologies under three traffic scenarios: normal, moderate, and congested conditions. The proposed model was compared with several baseline methods, including Pure DRL, DDPG, PPO, DQN, Multi-Agent DRL (MARL), PSO-only routing, Shortest Path First (SPF), and Equal Cost Multi-Path (ECMP). The results show that Hybrid DRL–PSO achieves the lowest latency values of 15.2 ms, 34.8 ms, and 55.3 ms, as well as the highest throughput values of 9.45 Mbps, 6.34 Mbps, and 4.27 Mbps across the three scenarios. In addition, the model maintains low packet loss rates of 0.05%, 1.2%, and 8.5%, and jitter values of 4.3 ms, 9.2 ms, and 16.6 ms, respectively. The main novelty of this work lies in integrating PSO as a pre-selection mechanism to generate K-best candidate paths, reducing the DRL action space and accelerating learning convergence for QoS-aware multipath routing. This hybrid approach also demonstrates the practical potential of combining learning-based intelligence and optimization techniques for adaptive traffic management in real-world SDN infrastructures.

Downloads

Download data is not yet available.

References

B. Isyaku, K. bin Abu Bakar, N. M. Yusuf, M. Abaker, A. Abdelmaboud, and W. Nagmeldin, “Software defined wireless sensor load balancing routing for internet of things applications: Review of approaches,” Heliyon, vol. 10, no. 9, p. e29965, May 2024, https://doi.org/10.1016/j.heliyon.2024.e29965

M. Taha, “An efficient software defined network controller-based routing adaptation for enhancing QoE of multimedia streaming service,” Multimed. Tools Appl., vol. 82, no. 22, pp. 33865–33888, Sep. 2023, https://doi.org/10.1007/s11042-023-14938-5

J. Li, J. Yoo, and J. W. Hong, “Dynamic control plane management for software‐defined networks,” International Journal of Network Management, vol. 26, no. 2, pp. 111–130, Mar. 2016, https://doi.org/10.1002/nem.1924

A. Hakiri, A. Gokhale, P. Berthou, D. C. Schmidt, and T. Gayraud, “Software-Defined Networking: Challenges and research opportunities for Future Internet,” Computer Networks, vol. 75, pp. 453–471, Dec. 2014, doi: 10.1016/j.comnet.2014.10.015.

M. Karakus and A. Durresi, “A survey: Control plane scalability issues and approaches in Software-Defined Networking (SDN),” Computer Networks, vol. 112, pp. 279–293, Jan. 2017, doi: 10.1016/j.comnet.2016.11.017.

G. Kim, Y. Kim, and H. Lim, “Deep Reinforcement Learning-Based Routing on Software-Defined Networks,” IEEE Access, vol. 10, pp. 18121–18133, 2022, https://doi.org/10.1109/ACCESS.2022.3151081

C. Liu, M. Xu, Y. Yang, and N. Geng, “DRL-OR: Deep Reinforcement Learning-based Online Routing for Multi-type Service Requirements,” in IEEE INFOCOM 2021 - IEEE Conference on Computer Communications, IEEE, May 2021, pp. 1–10. doi: 10.1109/INFOCOM42981.2021.9488736.

W. Liu, J. Cai, Q. C. Chen, and Y. Wang, “DRL-R: Deep reinforcement learning approach for intelligent routing in software-defined data-center networks,” Journal of Network and Computer Applications, vol. 177, p. 102865, Mar. 2021, doi: 10.1016/j.jnca.2020.102865.

S. Munikoti, D. Agarwal, L. Das, M. Halappanavar, and B. Natarajan, “Challenges and Opportunities in Deep Reinforcement Learning With Graph Neural Networks: A Comprehensive Review of Algorithms and Applications,” IEEE Trans. Neural Netw. Learn. Syst., vol. 35, no. 11, pp. 15051–15071, Nov. 2024, https://doi.org/10.1109/TNNLS.2023.3283523

J. Hao et al., “Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain,” IEEE Trans. Neural Netw. Learn. Syst., vol. 35, no. 7, pp. 8762–8782, Jul. 2024, https://doi.org/10.1109/TNNLS.2023.3236361

K. Ota, D. K. Jha, and A. Kanezaki, “A framework for training larger networks for deep Reinforcement learning,” Mach. Learn., vol. 113, no. 9, pp. 6115–6139, Sep. 2024, doi: 10.1007/s10994-024-06547-6.

H. Huangshui, F. Xinji, W. Chuhang, L. Ke, and G. Yuxin, “A Novel Particle Swarm Optimization-Based Clustering and Routing Protocol for Wireless Sensor Networks,” Wirel. Pers. Commun., vol. 133, no. 4, pp. 2175–2202, Dec. 2023, doi: 10.1007/s11277-024-10860-7.

S. Reong, H.-M. Wee, and Y.-L. Hsiao, “20 Years of Particle Swarm Optimization Strategies for the Vehicle Routing Problem: A Bibliometric Analysis,” Mathematics, vol. 10, no. 19, p. 3669, Oct. 2022, doi: 10.3390/math10193669.

P. Malarvizhi and G. Kavithaa, “Identifying Stable Path in Self Configured Networks Using Particle Swarm Optimization with Centrality Betweenness,” Journal of Electrical Engineering & Technology, vol. 19, no. 4, pp. 2701–2713, May 2024, doi: 10.1007/s42835-023-01741-3.

A. Omidvar and K. Mohammadi, “Particle swarm optimization in intelligent routing of delay-tolerant network routing,” EURASIP J. Wirel. Commun. Netw., vol. 2014, no. 1, p. 147, Dec. 2014, doi: 10.1186/1687-1499-2014-147.

Z. Zhang et al., “A Particle Swarm Optimization-Based Queue Scheduling and Optimization Mechanism for Large-Scale Low-Earth-Orbit Satellite Communication Networks,” Sensors, vol. 25, no. 4, p. 1069, Feb. 2025, doi: 10.3390/s25041069.

R. F. Ghani and L. Al-Jobouri, “Packet Loss Optimization in Router Forwarding Tasks Based on the Particle Swarm Algorithm,” Electronics (Basel)., vol. 12, no. 2, p. 462, Jan. 2023, doi: 10.3390/electronics12020462.

E. M. Roopa Devi, T. Hemalatha, D. Usha, and A. K. Nanda, “An optimal multipath routing protocol using hybrid gravitational search particle swarm optimization for secure communication,” International Journal of Communication Systems, vol. 37, no. 7, May 2024, doi: 10.1002/dac.5731.

X. Sun, Z. Jia, M. Zhao, and Z. Zhang, “Multipath Load Balancing in SDN/OSPF Hybrid Network,” in Network and Parallel Computing, G. R. Gao, D. Qian, X. Gao, B. Chapman, and W. Chen, Eds., Cham: Springer International Publishing, 2016, pp. 93–100, https://doi.org/10.1007/978-3-319-47099-3-8

Yi-Chih Lei, Kuochen Wang, and Yi-Huai Hsu, “Multipath routing in SDN-based Data Center Networks,” in 2015 European Conference on Networks and Communications (EuCNC), IEEE, Jun. 2015, pp. 365–369. doi: 10.1109/EuCNC.2015.7194100.

Y. Kulakov, A. Kohan, and Y. Hrabovenko, “Multipath Routing in Intelligent Transport Networks,” in Advances in Artificial Systems for Logistics Engineering, Z. Hu, Q. Zhang, S. Petoukhov, and M. He, Eds., Cham: Springer International Publishing, 2022, pp. 81–90, https://doi.org/10.1007/978-3-031-04809-8_7

V. Sai R., P. U., S. Ch., S. K. Panda, and G. V., “An Enhanced Dynamic Multi-Path Routing Algorithm for Software Defined Networks,” Procedia Comput. Sci., vol. 233, pp. 12–21, 2024, doi: 10.1016/j.procs.2024.03.191.

H. Hassen, S. Meherzi, and Z. Ben Jemaa, “Improved Exploration Strategy for Q-Learning Based Multipath Routing in SDN Networks,” Journal of Network and Systems Management, vol. 32, no. 2, p. 25, Apr. 2024, doi: 10.1007/s10922-024-09804-0.

T. Badageri, B. Hamdaoui, and R. Langar, “Load-Balanced Multipath Routing Through Software-Defined Networking,” in 2024 International Wireless Communications and Mobile Computing (IWCMC), IEEE, May 2024, pp. 1068–1073. doi: 10.1109/IWCMC61514.2024.10592316.

K. Rajkumar, M. Ramachandran, F. Al-Turjman, and R. Patan, “A reinforcement learning optimization for future smart cities using software defined networking,” International Journal of Machine Learning and Cybernetics, vol. 12, no. 11, pp. 3221–3233, Nov. 2021, doi: 10.1007/s13042-020-01245-w.

M. Jenabzadeh, V. Ayatollahitafti, M. Mollakhalili Meybodi, M. M. Ardakani, and A. Javadpour, “VNR_LBP: A New Approach to Congestion Control Using Virtualization and Switch Migration in SDN,” Wirel. Pers. Commun., vol. 139, no. 2, pp. 733–757, Nov. 2024, doi: 10.1007/s11277-024-11492-7.

T. Ma et al., “A hybrid congestion control scheme for Named Data Networking via SDN,” Discover Applied Sciences, vol. 6, no. 5, p. 246, May 2024, doi: 10.1007/s42452-024-05903-4.

M. A. Gunavathie and S. Umamaheswari, “Traffic-aware optimal routing in software defined networks by predicting traffic using neural network,” Expert Syst. Appl., vol. 239, p. 122415, Apr. 2024, doi: 10.1016/j.eswa.2023.122415.

Y. Wang, M. Othman, W. O. Choo, R. Liu, and X. Wang, “DFRDRL: a dynamic fuzzy routing algorithm based on deep reinforcement learning with guaranteed latency and bandwidth for software-defined networks,” J. Big Data, vol. 11, no. 1, p. 150, Oct. 2024, doi: 10.1186/s40537-024-01029-x.

C. Gao, H. Wang, F. Zhu, L. Zhai, and S. Yi, “A Particle Swarm Optimization Algorithm for Controller Placement Problem in Software Defined Network,” in Algorithms and Architectures for Parallel Processing, G. Wang, A. Zomaya, G. Martinez, and K. Li, Eds., Cham: Springer International Publishing, 2015, pp. 44–54, https://doi.org/10.1007/978-3-319-27137-8_4

Q. Zhang et al., “A new quantum particle swarm optimization algorithm for controller placement problem in software-defined networking,” Computers and Electrical Engineering, vol. 95, p. 107456, Oct. 2021, doi: 10.1016/j.compeleceng.2021.107456.

J. Chen, W. Xiao, H. Zhang, J. Zuo, and X. Li, “Dynamic routing optimization in software-defined networking based on a metaheuristic algorithm,” Journal of Cloud Computing, vol. 13, no. 1, p. 41, Feb. 2024, doi: 10.1186/s13677-024-00603-1.

Z. Duliński, G. Rzym, and P. Chołda, “MPLS-based reduction of flow table entries in SDN switches supporting multipath transmission,” Comput. Commun., vol. 151, pp. 365–385, Feb. 2020, doi: 10.1016/j.comcom.2019.12.052.

L. Zhang, Y. Lu, D. Zhang, H. Cheng, and P. Dong, “DSOQR: Deep Reinforcement Learning for Online QoS Routing in SDN-Based Networks,” Security and Communication Networks, vol. 2022, pp. 1–11, Nov. 2022, doi: 10.1155/2022/4457645.

Y. Lu, Y. Chen, X. Xu, Q. Fu, J. Chen, and L. Liu, “A Sub-flow Adaptive Multipath Routing Algorithm for Data Centre Network,” International Journal of Computational Intelligence Systems, vol. 16, no. 1, p. 25, Mar. 2023, doi: 10.1007/s44196-023-00199-5.

J. Zhang and R. O. Yahya, “DRL-based routing algorithm with guaranteed loss, latency and bandwidth in SDN networks: Application of online video conferencing,” Journal of King Saud University - Computer and Information Sciences, vol. 35, no. 10, p. 101805, Dec. 2023, doi: 10.1016/j.jksuci.2023.101805.

Downloads

Published

2026-04-26

Issue

Section

Computer Science Applications

How to Cite

Efendi, R. (2026). Hybrid Deep Reinforcement Learning and Particle Swarm Optimization for Accelerated Multipath Routing in Congested SDN Environments . Jurnal RESTI (Rekayasa Sistem Dan Teknologi Informasi), 10(2), 441-452. https://doi.org/10.29207/resti.v10i2.7324