Multi-layer　extreme　learning　machine　(ML-ELM)　is　a　stacked　extreme　learning　machine　based　auto-encoding　(ELM-AE).　It　provides　an　effective　solution　for　deep　feature　extraction　with　higher　training　efficiency.　To　enhance　the　local-input　invariance　of　feature　extraction,　we　propose　a　contractive　multi-layer　extreme　learning　machine　(C-MLELM)　by　adding　a　penalty　term　in　the　optimization　function　to　minimize　derivative　of　output　to　input　at　each　hidden　layer.　In　this　way,　the　extracted　feature　is　supposed　to　keep　consecutiveness　attribution　of　an　image.　The　experiments　have　been　done　on　MNIST　handwriting　dataset　and　face　expression　dataset　CAFEE.　The　results　show　that　it　outperforms　several　state-of-art　classification　algorithms　with　less　error　and　higher　training　efficiency.