A Principal Factor of Performance in Decoupled Front-End

Yuya DEGAWA; Toru KOIZUMI; Tomoki NAKAMURA; Ryota SHIOYA; Junichiro KADOMOTO; Hidetsugu IRIE; Shuichi SAKAI

doi:10.1587/transinf.2023PAP0011

IEICE TRANSACTIONS on Information

A Principal Factor of Performance in Decoupled Front-End

Yuya DEGAWA, Toru KOIZUMI, Tomoki NAKAMURA, Ryota SHIOYA, Junichiro KADOMOTO, Hidetsugu IRIE, Shuichi SAKAI

Full Text Views

0

Cite this

Summary :

One of the performance bottlenecks of a processor is the front-end that supplies instructions. Various techniques, such as cache replacement algorithms and hardware prefetching, have been investigated to facilitate smooth instruction supply at the front-end and to improve processor performance. In these approaches, one of the most important factors has been the reduction in the number of instruction cache misses. By using the number of instruction cache misses or derived factors, previous studies have explained the performance improvements achieved by their proposed methods. However, we found that the number of instruction cache misses does not always explain performance changes well in modern processors. This is because the front-end in modern processors handles subsequent instruction cache misses in overlap with earlier ones. Based on this observation, we propose a novel factor: the number of miss regions. We define a region as a sequence of instructions from one branch misprediction to the next, while we define a miss region as a region that contains one or more instruction cache misses. At the boundary of each region, the pipeline is flushed owing to a branch misprediction. Thus, cache misses after this boundary are not handled in overlap with cache misses before the boundary. As a result, the number of miss regions is equal to the number of cache misses that are processed without overlap. In this paper, we demonstrate that the number of miss regions can well explain the variation in performance through mathematical models and simulation results. The results show that the model explains cycles per instruction with an average error of 1.0% and maximum error of 4.1% when applying an existing prefetcher to the instruction cache. The idea of miss regions highlights that instruction cache misses and branch mispredictions interact with each other in processors with a decoupled front-end. We hope that considering this interaction will motivate the development of fast performance estimation methods and new microarchitectural methods.

Publication: IEICE TRANSACTIONS on Information Vol.E106-D No.12 pp.1960-1968

Publication Date: 2023/12/01

Publicized: 2023/06/30

Online ISSN: 1745-1361

DOI: 10.1587/transinf.2023PAP0011

Type of Manuscript: Special Section PAPER (Special Section on Forefront Computing)

Category

Authors

Yuya DEGAWA
  the University of Tokyo
Toru KOIZUMI
  the University of Tokyo
Tomoki NAKAMURA
  the University of Tokyo
Ryota SHIOYA
  the University of Tokyo
Junichiro KADOMOTO
  the University of Tokyo
Hidetsugu IRIE
  the University of Tokyo
Shuichi SAKAI
  the University of Tokyo

Keyword

instruction fetch, modeling techniques

Cite this

Copy

Yuya DEGAWA, Toru KOIZUMI, Tomoki NAKAMURA, Ryota SHIOYA, Junichiro KADOMOTO, Hidetsugu IRIE, Shuichi SAKAI, "A Principal Factor of Performance in Decoupled Front-End" in IEICE TRANSACTIONS on Information, vol. E106-D, no. 12, pp. 1960-1968, December 2023, doi: 10.1587/transinf.2023PAP0011.
Abstract: One of the performance bottlenecks of a processor is the front-end that supplies instructions. Various techniques, such as cache replacement algorithms and hardware prefetching, have been investigated to facilitate smooth instruction supply at the front-end and to improve processor performance. In these approaches, one of the most important factors has been the reduction in the number of instruction cache misses. By using the number of instruction cache misses or derived factors, previous studies have explained the performance improvements achieved by their proposed methods. However, we found that the number of instruction cache misses does not always explain performance changes well in modern processors. This is because the front-end in modern processors handles subsequent instruction cache misses in overlap with earlier ones. Based on this observation, we propose a novel factor: the number of miss regions. We define a region as a sequence of instructions from one branch misprediction to the next, while we define a miss region as a region that contains one or more instruction cache misses. At the boundary of each region, the pipeline is flushed owing to a branch misprediction. Thus, cache misses after this boundary are not handled in overlap with cache misses before the boundary. As a result, the number of miss regions is equal to the number of cache misses that are processed without overlap. In this paper, we demonstrate that the number of miss regions can well explain the variation in performance through mathematical models and simulation results. The results show that the model explains cycles per instruction with an average error of 1.0% and maximum error of 4.1% when applying an existing prefetcher to the instruction cache. The idea of miss regions highlights that instruction cache misses and branch mispredictions interact with each other in processors with a decoupled front-end. We hope that considering this interaction will motivate the development of fast performance estimation methods and new microarchitectural methods.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2023PAP0011/_p

Copy

@ARTICLE{e106-d_12_1960,
author={Yuya DEGAWA, Toru KOIZUMI, Tomoki NAKAMURA, Ryota SHIOYA, Junichiro KADOMOTO, Hidetsugu IRIE, Shuichi SAKAI, },
journal={IEICE TRANSACTIONS on Information},
title={A Principal Factor of Performance in Decoupled Front-End},
year={2023},
volume={E106-D},
number={12},
pages={1960-1968},
abstract={One of the performance bottlenecks of a processor is the front-end that supplies instructions. Various techniques, such as cache replacement algorithms and hardware prefetching, have been investigated to facilitate smooth instruction supply at the front-end and to improve processor performance. In these approaches, one of the most important factors has been the reduction in the number of instruction cache misses. By using the number of instruction cache misses or derived factors, previous studies have explained the performance improvements achieved by their proposed methods. However, we found that the number of instruction cache misses does not always explain performance changes well in modern processors. This is because the front-end in modern processors handles subsequent instruction cache misses in overlap with earlier ones. Based on this observation, we propose a novel factor: the number of miss regions. We define a region as a sequence of instructions from one branch misprediction to the next, while we define a miss region as a region that contains one or more instruction cache misses. At the boundary of each region, the pipeline is flushed owing to a branch misprediction. Thus, cache misses after this boundary are not handled in overlap with cache misses before the boundary. As a result, the number of miss regions is equal to the number of cache misses that are processed without overlap. In this paper, we demonstrate that the number of miss regions can well explain the variation in performance through mathematical models and simulation results. The results show that the model explains cycles per instruction with an average error of 1.0% and maximum error of 4.1% when applying an existing prefetcher to the instruction cache. The idea of miss regions highlights that instruction cache misses and branch mispredictions interact with each other in processors with a decoupled front-end. We hope that considering this interaction will motivate the development of fast performance estimation methods and new microarchitectural methods.},
keywords={},
doi={10.1587/transinf.2023PAP0011},
ISSN={1745-1361},
month={December},}

Copy

TY - JOUR
TI - A Principal Factor of Performance in Decoupled Front-End
T2 - IEICE TRANSACTIONS on Information
SP - 1960
EP - 1968
AU - Yuya DEGAWA
AU - Toru KOIZUMI
AU - Tomoki NAKAMURA
AU - Ryota SHIOYA
AU - Junichiro KADOMOTO
AU - Hidetsugu IRIE
AU - Shuichi SAKAI
PY - 2023
DO - 10.1587/transinf.2023PAP0011
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E106-D
IS - 12
JA - IEICE TRANSACTIONS on Information
Y1 - December 2023
AB - One of the performance bottlenecks of a processor is the front-end that supplies instructions. Various techniques, such as cache replacement algorithms and hardware prefetching, have been investigated to facilitate smooth instruction supply at the front-end and to improve processor performance. In these approaches, one of the most important factors has been the reduction in the number of instruction cache misses. By using the number of instruction cache misses or derived factors, previous studies have explained the performance improvements achieved by their proposed methods. However, we found that the number of instruction cache misses does not always explain performance changes well in modern processors. This is because the front-end in modern processors handles subsequent instruction cache misses in overlap with earlier ones. Based on this observation, we propose a novel factor: the number of miss regions. We define a region as a sequence of instructions from one branch misprediction to the next, while we define a miss region as a region that contains one or more instruction cache misses. At the boundary of each region, the pipeline is flushed owing to a branch misprediction. Thus, cache misses after this boundary are not handled in overlap with cache misses before the boundary. As a result, the number of miss regions is equal to the number of cache misses that are processed without overlap. In this paper, we demonstrate that the number of miss regions can well explain the variation in performance through mathematical models and simulation results. The results show that the model explains cycles per instruction with an average error of 1.0% and maximum error of 4.1% when applying an existing prefetcher to the instruction cache. The idea of miss regions highlights that instruction cache misses and branch mispredictions interact with each other in processors with a decoupled front-end. We hope that considering this interaction will motivate the development of fast performance estimation methods and new microarchitectural methods.
ER -

IEICE TRANSACTIONS on Information