For　autonomous　vehicles　driving　on　roads,　future　trajectories　of　surrounding　traffic　agents　(e.g.,　vehicles,　bicycles,　pedestrians)　are　essential　information.　The　prediction　of　future　trajectories　is　challenging　as　the　motion　of　traffic　agents　is　constantly　affected　by　spatial-temporal　interactions　from　agents　and　road　infrastructure.　To　take　those　interactions　into　account,　this　study　proposes　a　Graph　Attention　Transformer　(Gatformer)　in　which　a　traffic　scene　is　represented　by　a　sparse　graph.　To　maintain　the　spatial　and　temporal　information　of　traffic　agents　in　a　traffic　scene,　Convolutional　Neural　Networks　(CNNs)　are　utilized　to　extract　spatial　features　and　a　position　encoder　is　proposed　to　encode　the　spatial　features　and　the　corresponding　temporal　features.　Based　on　the　encoded　features,　a　Graph　Attention　Network　(GAT)　block　is　employed　to　model　the　agent-agent　and　agent-infrastructure　interactions　with　the　help　of　attention　mechanisms.　Finally,　a　Transformer　network　is　introduced　to　predict　trajectories　for　multiple　agents　simultaneously.　Experiments　are　conducted　over　the　Lyft　dataset　and　state-of-the-art　methods　are　introduced　for　comparison.　The　results　show　that　the　proposed　Gatformer　could　make　more　accurate　predictions　while　requiring　less　inference　time　than　its　counterparts.