Retinal　blood　vessel　morphological　abnormalities　are　generally　associated　with　cardiovascular,　cerebrovascular,　and　systemic　diseases,　automatic　artery/vein　classification　is　particularly　important　for　medical　image　analysis　and　clinical　decision　making.　This　work　proposes　a　retinal　artery/vein　classification　model　based　on　multi-scale　category　fusion　in　order　to　improve　the　accuracy　of　arteriovenous　classification.　Aiming　at　the　high　similarity　of　arteriovenous　features,　a　multi-scale　feature　extraction　module　is　proposed　to　enhance　feature　extraction　by　aggregating　multi-scale　features　of　hierarchical　residuals　in　Res2Net　single　residuals.　Furthermore,　a　multi-layer　semantic　supervision　structure　is　designed　to　supervise　and　fuse　the　arteriovenous　features　at　different　layers　to　obtain　more　semantic　details　to　enhance　the　distinguishing　ability　of　features.　Finally,　a　category-weighted　fusion　module　is　introduced　to　concatenate　the　feature　maps　of　the　same　category　together　to　refine　the　overall　segmentation　results.　The　proposed　method　is　verified　on　two　public　available　fundus　image　datasets　with　different　scales,　namely,　DRIVE　and　LES-AV.　The　experimental　results　show　that　the　proposed　method　performs　well　in　the　task　of　arteriovenous　classification　and　outperforms　most　of　the　existing　methods.