• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
搜索

Author:

Wang, Jindong (Wang, Jindong.) | Xu, Jinchao (Xu, Jinchao.) | Zhu, Jianqing (Zhu, Jianqing.)

Indexed by:

CPCI-S EI Scopus

Abstract:

Activation function plays an important role in neural networks. We propose to use hat activation function, namely the first order B-spline, as activation function for CNNs including MgNet and ResNet. Different from commonly used activation functions like ReLU, the hat function has a compact support and no obvious spectral bias. Although spectral bias is thought to be beneficial for generalization, we show that MgNet and ResNet with hat function still exhibit a slightly better generalization performance than CNNs with ReLU function by our experiments of classification on MNIST, CIFAR10/100 and ImageNet datasets. This indicates that CNNs without spectral bias can have a good generalization capability. We also illustrate that although hat function has a small activation area which is more likely to induce vanishing gradient problem, hat CNNs with various initialization methods still works well.

Keyword:

Spectral bias Convolutional neural network Activation function

Author Community:

  • [ 1 ] [Wang, Jindong]Peking Univ, Sch Math Sci, Beijing 100871, Peoples R China
  • [ 2 ] [Xu, Jinchao]Penn State Univ, Dept Math, University Pk, PA 16802 USA
  • [ 3 ] [Zhu, Jianqing]Beijing Univ Technol, Fac Sci, Beijing 100124, Peoples R China

Reprint Author's Address:

Show more details

Related Keywords:

Related Article:

Source :

COMPUTATIONAL SCIENCE, ICCS 2022, PT II

ISSN: 0302-9743

Year: 2022

Page: 319-327

Cited Count:

WoS CC Cited Count:

SCOPUS Cited Count:

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 1

Affiliated Colleges:

Online/Total:2698/10985755
Address:BJUT Library(100 Pingleyuan,Chaoyang District,Beijing 100124, China Post Code:100124) Contact Us:010-67392185
Copyright:BJUT Library Technical Support:Beijing Aegean Software Co., Ltd.