In　this　paper;　we　propose　a　novel　fast　mode　decision　scheme　for　intra　frame　and　inter　frame　in　the　H.264/AVC　video　coding　standard.　This　algorithm　only　computes　the　best　likely　modes　which　are　selected　by　the　value　of　variance　(second　moment).　The　variance　is　used　to　measure　the　smoothness　and　coarseness　of　the　texture,　and　then　a　few　modes　are　selected　according　to　the　value　of　variance.　The　proposed　algorithm　combines　the　variance　and　the　threshold,　which　can　remarkably　reduce　the　computational　complexity　and　improve　the　coding　efficiency.　Experimental　results　show　that　the　combination　of　the　proposed　algorithm　and　Jeon＇s　algorithm　can　reduce　encoding　time　by　52%　on　average,　whereas　the　PSNR　maintains　similar　and　the　bit　rate　has　a　tiny　increase.