Indexed by:
Abstract:
In this paper, it is proved that for one-hidden-layer ReLU networks all differentiable local minima are global inside each differentiable region. Necessary and sufficient conditions for the existences of differentiable local minima, saddle points and non-differentiable local minima are given, as well as their locations if they do exist. Building upon the theory, a linear programming based algorithm is designed to judge the existence of differentiable local minima, and is used to predict whether spurious local minima exist for the MNIST and CIFAR-10 datasets. Experimental results show that there are no spurious local minima for most typical weight vectors. These theoretical predictions are verified by demonstrating the consistency between them and the results of gradient descent search. ? 2021 Elsevier B.V. All rights reserved. Superscript/Subscript Available</comment
Keyword:
Reprint Author's Address:
Email:
Source :
KNOWLEDGE-BASED SYSTEMS
ISSN: 0950-7051
Year: 2021
Volume: 220
8 . 8 0 0
JCR@2022
ESI Discipline: COMPUTER SCIENCE;
ESI HC Threshold:87
JCR Journal Grade:1
Cited Count:
WoS CC Cited Count: 10
SCOPUS Cited Count: 13
ESI Highly Cited Papers on the List: 0 Unfold All
WanFang Cited Count:
Chinese Cited Count:
30 Days PV: 3
Affiliated Colleges: