In　real　world,　facial　expression　images　are　easily　disturbed　by　different　external　environments,　such　as　face　poses,　illumination　intensities,　and　occlusion,　and　features　extracted　from　whole　facial　expression　images　often　have　a　lot　of　noise,　which　leads　to　low　recognition　accuracy　of　the　model.　Inspired　by　the　facts　that　local　images　are　less　disturbed　by　the　environment　and　human　beings　usually　recognize　facial　expressions　by　integrating　some　local　information　about　human　faces,　this　paper　proposes　a　model　for　recognition　of　facial　expressions,　based　on　the　fusion　of　local　image　features,　that　aims　to　recognize　facial　expressions　by　comprehensively　considering　local　information　about　human　faces.　To　particularly　integrate　the　robust　characteristics　of　local　images,　this　paper　constructs　a　visual　self-attention　network　by　transformation　of　the　Transformer　model　in　the　field　of　natural　language　processing,　which　can　take　into　account　the　interaction　between　local　images　and　make　the　image　feature　more　discriminative.　To　demonstrate　the　performance　of　the　model,　this　paper　conducts　experiments　on　three　constrained　small　datasets,　CK+,　Oulu-CASIA　and　MMI,　and　three　unconstrained　large　datasets,　FER2013,　RAF-DB　and　Expw.　The　results　show　that　the　proposed　model　can　attain　the　recognition　accuracy　of　the　state-of-the-art　methods　on　all　datasets,　thereby　verifying　the　effectiveness　of　the　proposed　method.