• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
搜索

Author:

Song, Yang (Song, Yang.) | Li, Yunchun (Li, Yunchun.) | Yang, Hailong (Yang, Hailong.) | Xu, Jun (Xu, Jun.) | Luan, Zerong (Luan, Zerong.) | Li, Wei (Li, Wei.)

Indexed by:

EI Scopus SCIE CSCD

Abstract:

The data stream processing framework processes the stream data based on event-time to ensure that the request can be responded to in real-time. In reality, streaming data usually arrives out-of-order due to factors such as network delay. The data stream processing framework commonly adopts the watermark mechanism to address the data disorderedness. Watermark is a special kind of data inserted into the data stream with a timestamp, which helps the framework to decide whether the data received is late and thus be discarded. Traditional watermark generation strategies are periodic; they cannot dynamically adjust the watermark distribution to balance the responsiveness and accuracy. This paper proposes an adaptive watermark generation mechanism based on the time series prediction model to address the above limitation. This mechanism dynamically adjusts the frequency and timing of watermark distribution using the disordered data ratio and other lateness properties of the data stream to improve the system responsiveness while ensuring acceptable result accuracy. We implement the proposed mechanism on top of Flink and evaluate it with real-world datasets. The experiment results show that our mechanism is superior to the existing watermark distribution strategies in terms of both system responsiveness and result accuracy.

Keyword:

time series based prediction data stream processing dynamic adjustment watermark

Author Community:

  • [ 1 ] [Song, Yang]Beihang Univ, Sino German Joint Software Inst, Beijing 100191, Peoples R China
  • [ 2 ] [Li, Yunchun]Beihang Univ, Sino German Joint Software Inst, Beijing 100191, Peoples R China
  • [ 3 ] [Yang, Hailong]Beihang Univ, Sino German Joint Software Inst, Beijing 100191, Peoples R China
  • [ 4 ] [Li, Wei]Beihang Univ, Sino German Joint Software Inst, Beijing 100191, Peoples R China
  • [ 5 ] [Song, Yang]Beihang Univ, Sch Comp Sci & Engn, Beijing 100191, Peoples R China
  • [ 6 ] [Li, Yunchun]Beihang Univ, Sch Comp Sci & Engn, Beijing 100191, Peoples R China
  • [ 7 ] [Yang, Hailong]Beihang Univ, Sch Comp Sci & Engn, Beijing 100191, Peoples R China
  • [ 8 ] [Li, Wei]Beihang Univ, Sch Comp Sci & Engn, Beijing 100191, Peoples R China
  • [ 9 ] [Xu, Jun]Beijing Simulat Ctr, Sci & Technol Space Syst Simulat Lab, Beijing 100854, Peoples R China
  • [ 10 ] [Luan, Zerong]Beijing Univ Technol, Coll Life Sci & Bioengn, Beijing 100083, Peoples R China

Reprint Author's Address:

  • [Yang, Hailong]Beihang Univ, Sino German Joint Software Inst, Beijing 100191, Peoples R China;;[Yang, Hailong]Beihang Univ, Sch Comp Sci & Engn, Beijing 100191, Peoples R China

Show more details

Related Keywords:

Related Article:

Source :

FRONTIERS OF COMPUTER SCIENCE

ISSN: 2095-2228

Year: 2021

Issue: 6

Volume: 15

4 . 2 0 0

JCR@2022

ESI Discipline: COMPUTER SCIENCE;

ESI HC Threshold:87

JCR Journal Grade:2

Cited Count:

WoS CC Cited Count: 3

SCOPUS Cited Count: 3

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 10

Online/Total:348/10642655
Address:BJUT Library(100 Pingleyuan,Chaoyang District,Beijing 100124, China Post Code:100124) Contact Us:010-67392185
Copyright:BJUT Library Technical Support:Beijing Aegean Software Co., Ltd.