Emotion　recognition　from　macroexpression　and　microexpression　has　been　widely　used　in　applications　such　as　human-computer　interaction,　learning　status　evaluation,　and　mental　disorder　diagnosis.　However,　due　to　the　complexity　of　human　macroexpressions,　recognizing　macroexpressions　with　high　accuracy　is　a　challenging　task.　Moreover,　the　short　duration　and　low　movement　intensity　of　microexpressions　make　its　recognition　more　difficult.　For　MM-FER　(macro　and　microfacial　expression　recognition),　the　key　information　can　be　more　efficiently　expressed　by　a　graph.　In　this　article,　a　novel　framework　based　on　graph　neural　network　named　SSGNN　(spatial　and　spectral　domain　features　based　on　a　graph　neural　network)　is　designed　to　extract　spatial　and　spectral　domain　features　from　facial　images　for　MM-FER,　which　can　efficiently　recognize　both　macroexpressions　and　microexpressions　under　the　same　model.　SSGNN　consists　of　two　parts,　SPAGNN　and　SPEGNN,　which　are　used　to　extract　spectral　and　spatial　domain　features,　respectively.　Experiments　proved　that　jointly　using　the　spectral　and　spatial　information　extracted　by　SSGNN　can　largely　improve　the　performance　of　MM-FER　when　the　training　sample　is　limited.　First,　the　influences　of　different　neighbors　and　samples　to　the　model　performance　was　analyzed.　Then,　the　contribution　of　SPAGNN　and　SPEGNN　were　evaluated.　It　was　discovered　that　fusing　the　result　of　SPAGNN　and　SPEGNN　at　decision　level　further　improved　the　performance　of　MM-FER.　Experiment　proved　that　SSGNN　can　recognize　microexpression　acquired　by　various　sensors　with　higher　accuracy　under　different　image　resolutions　and　image　formats　than　the　compared　state-of-the-art　methods　in　most　cases.　A　cross-dataset　experiment　demonstrated　the　generalization　ability　of　SSGNN.