In　this　article,　theoretically,　it　is　shown　that　spurious　local　minima　are　common　for　deep　fully　connected　networks　and　average-pooling　convolutional　neural　networks　(CNNs)　with　piecewise　linear　activations　and　datasets　that　cannot　be　fit　by　linear　models.　Motivating　examples　are　given　to　explain　why　spurious　local　minima　exist:　each　output　neuron　of　deep　fully　connected　networks　and　CNNs　with　piecewise　linear　activations　produces　a　continuous　piecewise　linear　(CPWL)　function,　and　different　pieces　of　the　CPWL　output　can　optimally　fit　disjoint　groups　of　data　samples　when　minimizing　the　empirical　risk.　Fitting　data　samples　with　different　CPWL　functions　usually　results　in　different　levels　of　empirical　risk,　leading　to　the　prevalence　of　spurious　local　minima.　The　results　are　proved　in　general　settings　with　arbitrary　continuous　loss　functions　and　general　piecewise　linear　activations.　The　main　proof　technique　is　to　represent　a　CPWL　function　as　maximization　over　minimization　of　linear　pieces.　Deep　networks　with　piecewise　linear　activations　are　then　constructed　to　produce　these　linear　pieces　and　implement　the　maximization　over　minimization　operation.