As　the　performance　of　Energy　Management　Strategy　(EMS)　is　crucial　for　the　energy　efficiency　of　Hybrid　Electric　Vehicles　(HEVs),　a　Deep　Reinforcement　Learning　(DRL)-based　algorithm,　namely　Twin　Delayed　Deep　Deter-ministic　Policy　Gradient　(TD3),　is　adopted　to　design　EMS　for　the　power　Charge-Sustained　(CS)　stage　of　a　multi　-mode　plug-in　Hybrid　Electric　Vehicle　(HEV).　In　addition,　EMS　is　improved　by　combining　the　actor-network　of　TD3　with　Gumbel-Softmax　to　realize　mode　selection　and　torque　distribution　simultaneously,　which　is　a　discrete　(mode)-continuous　(engine　speed)　hybrid　action　space　and　not　applicable　in　original　TD3.　To　reduce　the　un-reasonable　exploration　of　agents　in　discrete　action,　a　rule-based　mode　control　mechanism　(RBMCM)　is　designed　and　involved　in　EMS.　The　improved　algorithm　speeds　up　the　learning　process　and　achieves　better　fuel　economy.　Simulation　results　show　that　the　gap　between　the　proposed　strategy　and　the　benchmark　dynamic　programming　(DP)　is　reduced　to　2.55%　in　the　selected　training　cycle.　Regarding　the　unknown　testing　cycles,　the　fuel　economy　of　agents　trained　by　the　improved　method　overperforms　traditional　DRL-based　EMS　when　it　reaches　more　than　90%　of　the　DP-based　benchmarking.　In　conclusion,　the　proposed　method　provides　a　theoretical　foundation　for　the　solution　of　the　hybrid　space　optimization　problem　for　hybrid　systems.