A Reinforcement Learning Approach for Admission Control in Mobile Multimedia Networks with Predictive Information

Jose Manuel GIMENEZ-GUZMAN; Jorge MARTINEZ-BAUSET; Vicent PLA

doi:10.1093/ietcom/e90-b.7.1663

A Reinforcement Learning Approach for Admission Control in Mobile Multimedia Networks with Predictive Information

Jose Manuel GIMENEZ-GUZMAN, Jorge MARTINEZ-BAUSET, Vicent PLA

Full Text Views

0

Cite this

Summary :

We study the problem of optimizing admission control policies in mobile multimedia cellular networks when predictive information regarding movement is available and we evaluate the gains that can be achieved by making such predictive information available to the admission controller. We consider a general class of prediction agents which forecast the number of future handovers and we evaluate the impact on performance of aspects like: whether the prediction refers to incoming and/or outgoing handovers, inaccurate predictions, the anticipation of the prediction and the way that predictions referred to different service classes are aggregated. For the optimization process we propose a novel Reinforcement Learning approach based on the concept of afterstates. The proposed approach, when compared with conventional Reinforcement Learning, yields better solutions and with higher precision. Besides it tackles more efficiently the curse of dimensionality inherent to multimedia scenarios. Numerical results show that the performance gains measured are higher when more specific information is provided about the handover time instants, i.e. when the anticipation time is deterministic instead of stochastic. It is also shown that the utilization of the network is maintained at very high values, even when the highest improvements are observed. We also compare an optimal policy obtained deploying our approach with a previously proposed heuristic prediction scheme, showing that plenty of room for technological innovation exists.

Publication: IEICE TRANSACTIONS on Communications Vol.E90-B No.7 pp.1663-1673

Publication Date: 2007/07/01

Publicized

Online ISSN: 1745-1345

DOI: 10.1093/ietcom/e90-b.7.1663

Type of Manuscript: PAPER

Category: Network

Cite this

Copy

Jose Manuel GIMENEZ-GUZMAN, Jorge MARTINEZ-BAUSET, Vicent PLA, "A Reinforcement Learning Approach for Admission Control in Mobile Multimedia Networks with Predictive Information" in IEICE TRANSACTIONS on Communications, vol. E90-B, no. 7, pp. 1663-1673, July 2007, doi: 10.1093/ietcom/e90-b.7.1663.
Abstract: We study the problem of optimizing admission control policies in mobile multimedia cellular networks when predictive information regarding movement is available and we evaluate the gains that can be achieved by making such predictive information available to the admission controller. We consider a general class of prediction agents which forecast the number of future handovers and we evaluate the impact on performance of aspects like: whether the prediction refers to incoming and/or outgoing handovers, inaccurate predictions, the anticipation of the prediction and the way that predictions referred to different service classes are aggregated. For the optimization process we propose a novel Reinforcement Learning approach based on the concept of afterstates. The proposed approach, when compared with conventional Reinforcement Learning, yields better solutions and with higher precision. Besides it tackles more efficiently the curse of dimensionality inherent to multimedia scenarios. Numerical results show that the performance gains measured are higher when more specific information is provided about the handover time instants, i.e. when the anticipation time is deterministic instead of stochastic. It is also shown that the utilization of the network is maintained at very high values, even when the highest improvements are observed. We also compare an optimal policy obtained deploying our approach with a previously proposed heuristic prediction scheme, showing that plenty of room for technological innovation exists.
URL: https://global.ieice.org/en_transactions/communications/10.1093/ietcom/e90-b.7.1663/_p

Copy

@ARTICLE{e90-b_7_1663,
author={Jose Manuel GIMENEZ-GUZMAN, Jorge MARTINEZ-BAUSET, Vicent PLA, },
journal={IEICE TRANSACTIONS on Communications},
title={A Reinforcement Learning Approach for Admission Control in Mobile Multimedia Networks with Predictive Information},
year={2007},
volume={E90-B},
number={7},
pages={1663-1673},
abstract={We study the problem of optimizing admission control policies in mobile multimedia cellular networks when predictive information regarding movement is available and we evaluate the gains that can be achieved by making such predictive information available to the admission controller. We consider a general class of prediction agents which forecast the number of future handovers and we evaluate the impact on performance of aspects like: whether the prediction refers to incoming and/or outgoing handovers, inaccurate predictions, the anticipation of the prediction and the way that predictions referred to different service classes are aggregated. For the optimization process we propose a novel Reinforcement Learning approach based on the concept of afterstates. The proposed approach, when compared with conventional Reinforcement Learning, yields better solutions and with higher precision. Besides it tackles more efficiently the curse of dimensionality inherent to multimedia scenarios. Numerical results show that the performance gains measured are higher when more specific information is provided about the handover time instants, i.e. when the anticipation time is deterministic instead of stochastic. It is also shown that the utilization of the network is maintained at very high values, even when the highest improvements are observed. We also compare an optimal policy obtained deploying our approach with a previously proposed heuristic prediction scheme, showing that plenty of room for technological innovation exists.},
keywords={},
doi={10.1093/ietcom/e90-b.7.1663},
ISSN={1745-1345},
month={July},}

Copy

TY - JOUR
TI - A Reinforcement Learning Approach for Admission Control in Mobile Multimedia Networks with Predictive Information
T2 - IEICE TRANSACTIONS on Communications
SP - 1663
EP - 1673
AU - Jose Manuel GIMENEZ-GUZMAN
AU - Jorge MARTINEZ-BAUSET
AU - Vicent PLA
PY - 2007
DO - 10.1093/ietcom/e90-b.7.1663
JO - IEICE TRANSACTIONS on Communications
SN - 1745-1345
VL - E90-B
IS - 7
JA - IEICE TRANSACTIONS on Communications
Y1 - July 2007
AB - We study the problem of optimizing admission control policies in mobile multimedia cellular networks when predictive information regarding movement is available and we evaluate the gains that can be achieved by making such predictive information available to the admission controller. We consider a general class of prediction agents which forecast the number of future handovers and we evaluate the impact on performance of aspects like: whether the prediction refers to incoming and/or outgoing handovers, inaccurate predictions, the anticipation of the prediction and the way that predictions referred to different service classes are aggregated. For the optimization process we propose a novel Reinforcement Learning approach based on the concept of afterstates. The proposed approach, when compared with conventional Reinforcement Learning, yields better solutions and with higher precision. Besides it tackles more efficiently the curse of dimensionality inherent to multimedia scenarios. Numerical results show that the performance gains measured are higher when more specific information is provided about the handover time instants, i.e. when the anticipation time is deterministic instead of stochastic. It is also shown that the utilization of the network is maintained at very high values, even when the highest improvements are observed. We also compare an optimal policy obtained deploying our approach with a previously proposed heuristic prediction scheme, showing that plenty of room for technological innovation exists.
ER -