In　this　work,　we　study　the　problem　of　separating　the　global　camera　motion　and　the　local　dynamic　motion　from　an　optical　flow.　Previous　methods　either　estimate　global　motions　by　a　parametric　model,　such　as　a　homography,　or　estimate　both　of　them　by　an　optical　flow　field.　However,　none　of　these　methods　can　directly　estimate　global　and　local　motions　through　an　end-to-end　manner.　In　addition,　separating　the　two　motions　accurately　from　a　hybrid　flow　field　is　challenging.　Because　one　motion　can　easily　confuse　the　estimate　of　the　other　one　when　they　are　compounded　together.　To　this　end,　we　propose　an　end-to-end　global　and　local　motion　estimation　network　GLM-Net.　We　design　two　encoder-decoder　structures　for　the　motion　separation　in　the　optical　flow　based　on　different　task　orientations.　One　structure　adopts　a　mask　autoencoder　to　extract　the　global　motion,　while　the　other　one　uses　attention　U-net　for　the　local　motion　refinement.　We　further　designed　two　effective　training　methods　to　overcome　the　problem　of　lacking　supervisions.　We　apply　our　method　on　the　action　recognition　datasets　NCAA　and　UCF-101　to　verify　the　accuracy　of　the　local　motion,　and　the　homography　estimation　dataset　DHE　for　the　accuracy　of　the　global　motion.　Experimental　results　show　that　our　method　can　achieve　competitive　performance　in　both　tasks　at　the　same　time,　validating　the　effectiveness　of　the　motion　separation.　©　2021　ACM.