With　the　rise　of　artificial　intelligence　in　recent　years,　the　field　of　object　recognition　is　making　rapid　progress.　Face　recognition　is　a　major　subarea　of　object　recognition　which　has　already　played　a　significant　role　in　our　life.　However,　despite　the　extensive　study　on　the　field　of　face　recognition,　video-based　face　recognition　is　still　a　tough　area　which　needs　further　research.　In　this　paper,　we　propose　a　model　based　on　deep　convolutional　network　for　video-based　face　recognition.　Our　model　split　video　images　into　two　sets,　a　set　of　key　frames　and　the　other　set　is　made　up　with　non-keys,　for　different　tasks　to　lower　the　computational　complexity　of　the　model.　Besides,　we　introduce　spatial　pyramid　pooling　and　center　loss　to　our　method　for　classification　task.　Our　method　presented　in　this　paper　reached　an　accuracy　of　96.06%　on　YouTube　Faces　dataset.　The　results　indicate　our　approach　possesses　high　precision　as　well　as　a　strong　real-time　performance.