IEICE TRANSACTIONS on Information
A Lightweight Reinforcement Learning Based Packet Routing Method Using Online Sequential Learning
Summary :
Existing simple routing protocols (e.g., OSPF, RIP) have some disadvantages of being inflexible and prone to congestion due to the concentration of packets on particular routers. To address these issues, packet routing methods using machine learning have been proposed recently. Compared to these algorithms, machine learning based methods can choose a routing path intelligently by learning efficient routes. However, machine learning based methods have a disadvantage of training time overhead. We thus focus on a lightweight machine learning algorithm, OS-ELM (Online Sequential Extreme Learning Machine), to reduce the training time. Although previous work on reinforcement learning using OS-ELM exists, it has a problem of low learning accuracy. In this paper, we propose OS-ELM QN (Q-Network) with a prioritized experience replay buffer to improve the learning performance. It is compared to a deep reinforcement learning based packet routing method using a network simulator. Experimental results show that introducing the experience replay buffer improves the learning performance. OS-ELM QN achieves a 2.33 times speedup than a DQN (Deep Q-Network) in terms of learning speed. Regarding the packet transfer latency, OS-ELM QN is comparable or slightly inferior to the DQN while they are better than OSPF in most cases since they can distribute congestions.
- Publication
- IEICE TRANSACTIONS on Information Vol.E106-D No.11 pp.1796-1807
- Publication Date
- 2023/11/01
- Publicized
- 2023/08/15
- Online ISSN
- 1745-1361
- DOI
- 10.1587/transinf.2022EDP7231
- Type of Manuscript
- PAPER
- Category
- Computer System
Authors
Kenji NEMOTO
Keio University
Hiroki MATSUTANI
Keio University
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Kenji NEMOTO, Hiroki MATSUTANI, "A Lightweight Reinforcement Learning Based Packet Routing Method Using Online Sequential Learning" in IEICE TRANSACTIONS on Information,
vol. E106-D, no. 11, pp. 1796-1807, November 2023, doi: 10.1587/transinf.2022EDP7231.
Abstract: Existing simple routing protocols (e.g., OSPF, RIP) have some disadvantages of being inflexible and prone to congestion due to the concentration of packets on particular routers. To address these issues, packet routing methods using machine learning have been proposed recently. Compared to these algorithms, machine learning based methods can choose a routing path intelligently by learning efficient routes. However, machine learning based methods have a disadvantage of training time overhead. We thus focus on a lightweight machine learning algorithm, OS-ELM (Online Sequential Extreme Learning Machine), to reduce the training time. Although previous work on reinforcement learning using OS-ELM exists, it has a problem of low learning accuracy. In this paper, we propose OS-ELM QN (Q-Network) with a prioritized experience replay buffer to improve the learning performance. It is compared to a deep reinforcement learning based packet routing method using a network simulator. Experimental results show that introducing the experience replay buffer improves the learning performance. OS-ELM QN achieves a 2.33 times speedup than a DQN (Deep Q-Network) in terms of learning speed. Regarding the packet transfer latency, OS-ELM QN is comparable or slightly inferior to the DQN while they are better than OSPF in most cases since they can distribute congestions.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2022EDP7231/_p
Copy
@ARTICLE{e106-d_11_1796,
author={Kenji NEMOTO, Hiroki MATSUTANI, },
journal={IEICE TRANSACTIONS on Information},
title={A Lightweight Reinforcement Learning Based Packet Routing Method Using Online Sequential Learning},
year={2023},
volume={E106-D},
number={11},
pages={1796-1807},
abstract={Existing simple routing protocols (e.g., OSPF, RIP) have some disadvantages of being inflexible and prone to congestion due to the concentration of packets on particular routers. To address these issues, packet routing methods using machine learning have been proposed recently. Compared to these algorithms, machine learning based methods can choose a routing path intelligently by learning efficient routes. However, machine learning based methods have a disadvantage of training time overhead. We thus focus on a lightweight machine learning algorithm, OS-ELM (Online Sequential Extreme Learning Machine), to reduce the training time. Although previous work on reinforcement learning using OS-ELM exists, it has a problem of low learning accuracy. In this paper, we propose OS-ELM QN (Q-Network) with a prioritized experience replay buffer to improve the learning performance. It is compared to a deep reinforcement learning based packet routing method using a network simulator. Experimental results show that introducing the experience replay buffer improves the learning performance. OS-ELM QN achieves a 2.33 times speedup than a DQN (Deep Q-Network) in terms of learning speed. Regarding the packet transfer latency, OS-ELM QN is comparable or slightly inferior to the DQN while they are better than OSPF in most cases since they can distribute congestions.},
keywords={},
doi={10.1587/transinf.2022EDP7231},
ISSN={1745-1361},
month={November},}
Copy
TY - JOUR
TI - A Lightweight Reinforcement Learning Based Packet Routing Method Using Online Sequential Learning
T2 - IEICE TRANSACTIONS on Information
SP - 1796
EP - 1807
AU - Kenji NEMOTO
AU - Hiroki MATSUTANI
PY - 2023
DO - 10.1587/transinf.2022EDP7231
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E106-D
IS - 11
JA - IEICE TRANSACTIONS on Information
Y1 - November 2023
AB - Existing simple routing protocols (e.g., OSPF, RIP) have some disadvantages of being inflexible and prone to congestion due to the concentration of packets on particular routers. To address these issues, packet routing methods using machine learning have been proposed recently. Compared to these algorithms, machine learning based methods can choose a routing path intelligently by learning efficient routes. However, machine learning based methods have a disadvantage of training time overhead. We thus focus on a lightweight machine learning algorithm, OS-ELM (Online Sequential Extreme Learning Machine), to reduce the training time. Although previous work on reinforcement learning using OS-ELM exists, it has a problem of low learning accuracy. In this paper, we propose OS-ELM QN (Q-Network) with a prioritized experience replay buffer to improve the learning performance. It is compared to a deep reinforcement learning based packet routing method using a network simulator. Experimental results show that introducing the experience replay buffer improves the learning performance. OS-ELM QN achieves a 2.33 times speedup than a DQN (Deep Q-Network) in terms of learning speed. Regarding the packet transfer latency, OS-ELM QN is comparable or slightly inferior to the DQN while they are better than OSPF in most cases since they can distribute congestions.
ER -
English
