• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
搜索

Author:

Tao, Qinghua (Tao, Qinghua.) | Xu, Jun (Xu, Jun.) | Li, Zhen (Li, Zhen.) | Xie, Na (Xie, Na.) | Wang, Shuning (Wang, Shuning.) | Li, Xiaoli (Li, Xiaoli.) (Scholars:李晓理) | Suykens, Johan A. K. (Suykens, Johan A. K..)

Indexed by:

EI Scopus SCIE

Abstract:

The adaptive hinging hyperplane (AHH) model is a popular piecewise linear representation with a generalized tree structure and has been successfully applied in dynamic system identification. In this article, we aim to construct the deep AHH (DAHH) model to extend and generalize the networking of AHH model for high-dimensional problems. The network structure of DAHH is determined through a forward growth, in which the activity ratio is introduced to select effective neurons and no connecting weights are involved between the layers. Then, all neurons in the DAHH network can be flexibly connected to the output in a skip-layer format, and only the corresponding weights are the parameters to optimize. With such a network framework, the backpropagation algorithm can be implemented in DAHH to efficiently tackle large-scale problems and the gradient vanishing problem is not encountered in the training of DAHH. In fact, the optimization problem of DAHH can maintain convexity with convex loss in the output layer, which brings natural advantages in optimization. Different from the existing neural networks, DAHH is easier to interpret, where neurons are connected sparsely and analysis of variance (ANOVA) decomposition can be applied, facilitating to revealing the interactions between variables. A theoretical analysis toward universal approximation ability and explicit domain partitions are also derived. Numerical experiments verify the effectiveness of the proposed DAHH.

Keyword:

piecewise linear (PWL) Neurons Topology Adaptive systems Adaptive hinging hyperplanes (AHHs) analysis of variance (ANOVA) decomposition skip-layer connection domain partition Artificial neural networks Network topology Training Optimization

Author Community:

  • [ 1 ] [Tao, Qinghua]Katholieke Univ Leuven, STADIUS, ESAT, B-3001 Leuven, Belgium
  • [ 2 ] [Suykens, Johan A. K.]Katholieke Univ Leuven, STADIUS, ESAT, B-3001 Leuven, Belgium
  • [ 3 ] [Tao, Qinghua]Tsinghua Univ, Dept Automat, Beijing 100084, Peoples R China
  • [ 4 ] [Wang, Shuning]Tsinghua Univ, Dept Automat, Beijing 100084, Peoples R China
  • [ 5 ] [Xu, Jun]Harbin Inst Technol, Sch Mech Engn & Automat, Shenzhen 518055, Peoples R China
  • [ 6 ] [Li, Zhen]Harbin Inst Technol, Sch Mech Engn & Automat, Shenzhen 518055, Peoples R China
  • [ 7 ] [Xie, Na]Cent Univ Finance & Econ, Sch Management Sci & Engn, Beijing 100081, Peoples R China
  • [ 8 ] [Li, Xiaoli]Beijing Univ Technol, Fac Informat Technol, Beijing 100124, Peoples R China

Reprint Author's Address:

Show more details

Related Keywords:

Source :

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS

ISSN: 2162-237X

Year: 2021

Issue: 11

Volume: 33

Page: 6373-6387

1 0 . 4 0 0

JCR@2022

ESI Discipline: COMPUTER SCIENCE;

ESI HC Threshold:87

JCR Journal Grade:1

Cited Count:

WoS CC Cited Count: 5

SCOPUS Cited Count: 6

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 2

Affiliated Colleges:

Online/Total:509/10600948
Address:BJUT Library(100 Pingleyuan,Chaoyang District,Beijing 100124, China Post Code:100124) Contact Us:010-67392185
Copyright:BJUT Library Technical Support:Beijing Aegean Software Co., Ltd.