In　order　to　quickly　help　lifesavers　judge　whether　people　are　drowning　in　the　swimming　pool,　this　paper　proposes　one　efficient　behavior　recognition　approach　by　means　of　video　sequences　of　underwater.　First,　by　analyzing　the　spatial　distribution　of　swimming　pool　when　swimmers　are　normally　swimming,　the　data　labeling　and　swimmer　detection　methods　are　determined.　Second,　a　behavior　recognition　framework　of　swimmers　on　the　basis　of　YOLOv4　algorithm　(BR-YOLOv4)　is　proposed　in　this　paper.　The　spatial　relationship　between　the　location　information　of　the　target　and　swimming/drowning　area　of　swimming　pool　is　analyzed　to　further　determine　the　swimmer’s　drowning　or　swimming　behavior.　This　paper　compares　the　detection　accuracy　of　different　detection　algorithms　and　analyzes　the　detection　effect　of　different　pool　angles　and　different　swimmer　densities.　Test　results　show　that　the　mean　precision　rate　of　drowning　is　94.62%,　the　mean　false　rate　is　1.43%　,　and　the　mean　missing　rate　is　3.57%.　The　mean　precision　rate　of　swimming　is　97.86%,　the　mean　false　rate　is　7.93%,　the　mean　missing　rate　is　5.93%　,　and　the　average　frame　rate　is　33f/s.　All　the　results　show　that　the　method　proposed　in　this　paper　meets　the　real-time　detection　requirements　and　does　well　in　swimmer　behavior　recognition　and　provides　technical　support　for　reducing　drowning　accidents　in　public　swimming　pools.　©　2022,　The　Author(s),　under　exclusive　licence　to　Springer-Verlag　London　Ltd.,　part　of　Springer　Nature.