The　effective　segmentation　of　the　lesion　region　in　gastric　cancer　images　can　assist　physicians　in　diagnosing　and　reducing　the　probability　of　misdiagnosis.　The　U-Net　has　been　proven　to　provide　segmentation　results　comparable　to　specialists　in　medical　image　segmentation　because　of　its　ability　to　extract　high-level　semantic　information.　However,　it　has　limitations　in　obtaining　global　contextual　information.　On　the　other　hand,　the　Transformer　excels　at　modeling　explicit　long-range　relations　but　cannot　capture　low-level　detail　information.　Hence,　this　paper　proposes　a　Dual-Branch　Hybrid　Network　based　on　the　fusion　Transformer　and　U-Net　to　overcome　both　limitations.　We　propose　the　Deep　Feature　Aggregation　Decoder　(DFA)　by　aggregating　only　the　in-depth　features　to　obtain　salient　lesion　features　for　both　branches　and　reduce　the　complexity　of　the　model.　Besides,　we　design　a　Feature　Fusion　(FF)　module　utilizing　the　multi-modal　fusion　mechanisms　to　interact　with　independent　features　of　various　modalities　and　the　linear　Hadamard　product　to　fuse　the　feature　information　extracted　from　both　branches.　Finally,　the　Transformer　loss,　the　U-Net　loss,　and　the　fused　loss　are　compared　to　the　ground　truth　label　for　joint　training.　Experimental　results　show　that　our　proposed　method　has　an　IOU　of　81.3%,　a　Dice　coefficient　of　89.5%,　and　an　Accuracy　of　94.0%.　These　metrics　demonstrate　that　our　model　outperforms　the　existing　models　in　obtaining　high-quality　segmentation　results,　which　has　excellent　potential　for　clinical　analysis　and　diagnosis.　The　code　and　implementation　details　are　available　at　Github,　https://github.com/ZYY01/DBH-Net/.