The search functionality is under construction.
The search functionality is under construction.

Eager Memory Management for In-Memory Data Analytics

Hakbeom JANG, Jonghyun BAE, Tae Jun HAM, Jae W. LEE

  • Full Text Views

    0

  • Cite this
Errata[Uploaded on April 1,2019]

Summary :

This paper introduces e-spill, an eager spill mechanism, which dynamically finds the optimal spill-threshold by monitoring the GC time at runtime and thereby prevent expensive GC overhead. Our e-spill adopts a slow-start model to gradually increase the spill-threshold until it reaches the optimal point without substantial GCs. We prototype e-spill as an extension to Spark and evaluate it using six workloads on three different parallel platforms. Our evaluations show that e-spill improves performance by up to 3.80× and saves the cost of cluster operation on Amazon EC2 cloud by up to 51% over the baseline system following Spark Tuning Guidelines.

Publication
IEICE TRANSACTIONS on Information Vol.E102-D No.3 pp.632-636
Publication Date
2019/03/01
Publicized
2018/12/11
Online ISSN
1745-1361
DOI
10.1587/transinf.2018EDL8199
Type of Manuscript
LETTER
Category
Computer System

Authors

Hakbeom JANG
  Sungkyunkwan University
Jonghyun BAE
  Seoul National University
Tae Jun HAM
  Seoul National University
Jae W. LEE
  Seoul National University

Keyword