• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
搜索

Author:

Liang, Y. (Liang, Y..) | Chen, K. (Chen, K..) | Yi, L. (Yi, L..) | Su, X. (Su, X..) | Jin, X. (Jin, X..)

Indexed by:

EI Scopus SCIE

Abstract:

Complex data-parallel job contains task dependency information defined as Directed Acyclic Graph (DAG). For convenience, the DAG presented data-parallel jobs are named as DAG jobs. The prevalence of DAG jobs in modern data centers has made the scheduling oriented job characterization a big challenge. This paper proposes a deep graph-temporal clustering framework, i.e., DeGTeC, to efficiently categorize DAG jobs leveraging the graphic and temporal information in DAGs. The categorization result can then be naturally used to characterize the resource consumption pattern of DAG jobs. The DeGTeC framework is constructed mainly based on two autoencoders, i.e., TaskAE and JobAE. TaskAE and JobAE contain spectral graph convolutional network (GCN) layers, temporal convolutional network (TCN) layers, and the adaptive pooling layers to help build task embeddings and job embeddings. An extra embedding sorting step takes in the sequential order information and the depth-bias information for job clustering. To our best knowledge, DeGTeC is the first solution to do resource consumption characterization of DAG jobs fully leveraging the task dependencies defined in DAG. Experimental results demonstrate that the DeGTeC framework outperforms the state-of-the-art job resource consumption characterization methods. © 2022 Elsevier B.V.

Keyword:

Clustering Autoencoder Graph convolutional network Temporal convolutional network Data-parallel job Data center

Author Community:

  • [ 1 ] [Liang Y.]Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China
  • [ 2 ] [Chen K.]Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China
  • [ 3 ] [Yi L.]Yunhe Enmo (Beijing) Information Center, Beijing, 100080, China
  • [ 4 ] [Su X.]Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China
  • [ 5 ] [Jin X.]Zhuoxi Brain and Intelligence Research Institute, Hanzhou, 311121, China

Reprint Author's Address:

Email:

Show more details

Related Keywords:

Source :

Future Generation Computer Systems

ISSN: 0167-739X

Year: 2023

Volume: 141

Page: 81-95

7 . 5 0 0

JCR@2022

ESI Discipline: COMPUTER SCIENCE;

ESI HC Threshold:19

Cited Count:

WoS CC Cited Count:

SCOPUS Cited Count: 7

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 0

Affiliated Colleges:

Online/Total:410/10633637
Address:BJUT Library(100 Pingleyuan,Chaoyang District,Beijing 100124, China Post Code:100124) Contact Us:010-67392185
Copyright:BJUT Library Technical Support:Beijing Aegean Software Co., Ltd.