The search functionality is under construction.
The search functionality is under construction.

A Lightweight Reinforcement Learning Based Packet Routing Method Using Online Sequential Learning

Kenji NEMOTO, Hiroki MATSUTANI

  • Full Text Views

    0

  • Cite this

Summary :

Existing simple routing protocols (e.g., OSPF, RIP) have some disadvantages of being inflexible and prone to congestion due to the concentration of packets on particular routers. To address these issues, packet routing methods using machine learning have been proposed recently. Compared to these algorithms, machine learning based methods can choose a routing path intelligently by learning efficient routes. However, machine learning based methods have a disadvantage of training time overhead. We thus focus on a lightweight machine learning algorithm, OS-ELM (Online Sequential Extreme Learning Machine), to reduce the training time. Although previous work on reinforcement learning using OS-ELM exists, it has a problem of low learning accuracy. In this paper, we propose OS-ELM QN (Q-Network) with a prioritized experience replay buffer to improve the learning performance. It is compared to a deep reinforcement learning based packet routing method using a network simulator. Experimental results show that introducing the experience replay buffer improves the learning performance. OS-ELM QN achieves a 2.33 times speedup than a DQN (Deep Q-Network) in terms of learning speed. Regarding the packet transfer latency, OS-ELM QN is comparable or slightly inferior to the DQN while they are better than OSPF in most cases since they can distribute congestions.

Publication
IEICE TRANSACTIONS on Information Vol.E106-D No.11 pp.1796-1807
Publication Date
2023/11/01
Publicized
2023/08/15
Online ISSN
1745-1361
DOI
10.1587/transinf.2022EDP7231
Type of Manuscript
PAPER
Category
Computer System

Authors

Kenji NEMOTO
  Keio University
Hiroki MATSUTANI
  Keio University

Keyword