• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
搜索

Author:

Liu, Bo (Liu, Bo.) (Scholars:刘博)

Indexed by:

EI Scopus SCIE

Abstract:

In this paper, it is proved that for one-hidden-layer ReLU networks all differentiable local minima are global inside each differentiable region. Necessary and sufficient conditions for the existences of differentiable local minima, saddle points and non-differentiable local minima are given, as well as their locations if they do exist. Building upon the theory, a linear programming based algorithm is designed to judge the existence of differentiable local minima, and is used to predict whether spurious local minima exist for the MNIST and CIFAR-10 datasets. Experimental results show that there are no spurious local minima for most typical weight vectors. These theoretical predictions are verified by demonstrating the consistency between them and the results of gradient descent search. ? 2021 Elsevier B.V. All rights reserved. Superscript/Subscript Available</comment

Keyword:

ReLU networks Deep learning theory Saddle points Local minima Loss landscape

Author Community:

  • [ 1 ] [Liu, Bo]Beijing Univ Technol, Fac Informat Technol, Coll Comp Sci, Beijing, Peoples R China

Reprint Author's Address:

  • 刘博

    [Liu, Bo]Beijing Univ Technol, Fac Informat Technol, Coll Comp Sci, Beijing, Peoples R China

Show more details

Related Keywords:

Source :

KNOWLEDGE-BASED SYSTEMS

ISSN: 0950-7051

Year: 2021

Volume: 220

8 . 8 0 0

JCR@2022

ESI Discipline: COMPUTER SCIENCE;

ESI HC Threshold:87

JCR Journal Grade:1

Cited Count:

WoS CC Cited Count: 10

SCOPUS Cited Count: 13

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 3

Online/Total:562/10608586
Address:BJUT Library(100 Pingleyuan,Chaoyang District,Beijing 100124, China Post Code:100124) Contact Us:010-67392185
Copyright:BJUT Library Technical Support:Beijing Aegean Software Co., Ltd.