The search functionality is under construction.
The search functionality is under construction.

Author Search Result

[Author] Filip DE TURCK(1hit)

1-1hit
  • Managing the Synchronization in the Lambda Architecture for Optimized Big Data Analysis Open Access

    Thomas VANHOVE  Gregory VAN SEGHBROECK  Tim WAUTERS  Bruno VOLCKAERT  Filip DE TURCK  

     
    INVITED PAPER

      Vol:
    E99-B No:2
      Page(s):
    297-306

    In a world of continuously expanding amounts of data, retrieving interesting information from enormous data sets becomes more complex every day. Solutions for precomputing views on these big data sets mostly follow either an offline approach, which is slow but can take into account the entire data set, or a streaming approach, which is fast but only relies on the latest data entries. A hybrid solution was introduced through the Lambda architecture concept. It combines both offline and streaming approaches by analyzing data in a fast speed layer first, and in a slower batch layer later. However, this introduces a new synchronization challenge: once the data is analyzed by the batch layer, the corresponding information needs to be removed in the speed layer without introducing redundancy or loss of data. In this paper we propose a new approach to implement the Lambda architecture concept independent of the technologies used for offline and stream computing. A universal solution is provided to manage the complex synchronization introduced by the Lambda architecture and techniques to provide fault tolerance. The proposed solution is evaluated by means of detailed experimental results.