• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
搜索

Author:

Zhu, Xin (Zhu, Xin.) | Geng, Shuqin (Geng, Shuqin.) | Hou, Yupeng (Hou, Yupeng.) | Peng, Xiaohong (Peng, Xiaohong.) | Hou, Ligang (Hou, Ligang.) | Zhang, Mingming (Zhang, Mingming.) | Chu, Menghao (Chu, Menghao.)

Indexed by:

EI Scopus

Abstract:

In this article, we design and implement an air quality data processing system based on big data technology, which leverages Flume to acquire data, Kafka to cache data, and Storm to process data, and finally save the results to the in-memory database called Redis and distributed database called HBase. In Redis, only the latest data of each monitoring point is saved, and the original data is deleted periodically to reduce the memory load. An application is implemented by which people can get real-time data from Redis and query historical data from HBase. Through experiments, it is proved that querying real-time data from Redis is faster than relational database. The system solves the issues of poor storage capacity and slow query speed when using relational database in the big data environment, and improves the efficiency of data processing significantly. © 2018 IEEE.

Keyword:

Big data Relational database systems Search engines Air pollution Query processing Distributed database systems Data handling Storms Air quality Cache memory

Author Community:

  • [ 1 ] [Zhu, Xin]Department of Electrical and Computer Engineering, Beijing University of Technology, BJUT, Beijing; 100124, China
  • [ 2 ] [Geng, Shuqin]Department of Electrical and Computer Engineering, Beijing University of Technology, BJUT, Beijing; 100124, China
  • [ 3 ] [Hou, Yupeng]Department of Electrical and Computer Engineering, Beijing University of Technology, BJUT, Beijing; 100124, China
  • [ 4 ] [Peng, Xiaohong]Department of Electrical and Computer Engineering, Beijing University of Technology, BJUT, Beijing; 100124, China
  • [ 5 ] [Hou, Ligang]Department of Electrical and Computer Engineering, Beijing University of Technology, BJUT, Beijing; 100124, China
  • [ 6 ] [Zhang, Mingming]Department of Electrical and Computer Engineering, Beijing University of Technology, BJUT, Beijing; 100124, China
  • [ 7 ] [Chu, Menghao]Department of Electrical and Computer Engineering, Beijing University of Technology, BJUT, Beijing; 100124, China

Reprint Author's Address:

Email:

Show more details

Related Keywords:

Related Article:

Source :

Year: 2018

Page: 1846-1850

Language: English

Cited Count:

WoS CC Cited Count: 0

SCOPUS Cited Count:

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 9

Online/Total:1075/10532195
Address:BJUT Library(100 Pingleyuan,Chaoyang District,Beijing 100124, China Post Code:100124) Contact Us:010-67392185
Copyright:BJUT Library Technical Support:Beijing Aegean Software Co., Ltd.