Giselle van Dongen at IEEE Big Data 2018 Congress in SF
Giselle van Dongen (PhD student & Data Scientist at Klarrio), with onsite presence by Professor Van den Poel, was presenting the paper “Latency Measurement of Fine-Grained Operations in Benchmarking Distributed Stream Processing Frameworks” at the IEEE Big Data 2018 Congress in San Francisco.
This ‘Work in Progress’ Paper, is an initial step towards a full publication on benchmarking streaming frameworks, a research project from Klarrio, in close collaboration with UGent Big Data Analytics Team.
This paper describes a benchmark for streaming frameworks allowing accurate latency benchmarking of fine-grained individual stages of a processing pipeline. By determining the latency of distinct common operations in the processing flow instead of the end-to-end latency, we can form guidelines for efficient processing pipeline design. In this work, we use Apache Flink to validate the benchmark setup. The full paper will be published later this year.
Read the ‘Work in Progress’ Paper