Rainy　weather　conditions　significantly　degrade　image　quality,　posing　a　major　challenge　for　object　detection　tasks.　Conventional　methods　often　address　this　issue　through　domain　adaptation,　or　the　＇derain　then　detect＇　approach　that　utilizes　image　deraining　as　the　preprocessing　technique.　This　paper　presents　KBY-Net,　a　novel　end-to-end　Y-Net　architecture　that　is　built　upon　the　YOLOv8　architecture　and　leverages　multi-task　learning　for　concurrent　image　restoration　and　object　detection.　First,　KBY-Net　incorporates　a　novel　KBY-decoder　designed　for　image　deraining.　This　decoder　leverages　Cross　Stage　Partial　(CSP)　layer　and　kernel　basis　attention　(KBA)　module　to　improve　feature　representation.　Second,　KBY-Net　adopted　two　innovative　modules;　a　multi-Dconv　head　transposed　attention　(MDTA)　module　at　the　bottleneck　and　a　multi-axis　feature　fusion　(MFF)　block　at　the　neck　of　the　Y-Net.　The　multi-DConv　module　empowers　the　model　to　capture　long-range　dependencies　and　complex　representations,　and　the　MFF　block　refines　the　extracted　features　-　both　contribute　significantly　to　accurate　object　detection　in　challenging　rainy　scenes.　Empirical　evaluations　on　benchmark　rainy　datasets　demonstrate　that　KBY-Net　outperforms　the　state-ofthe-art　object　detection　approaches　by　a　significant　margin　both　quantitatively　and　qualitatively.　©　2024　Copyright　held　by　the　owner/author(s).