The　bus　passenger　data　are　very　important　for　urban　bus　dispatching　management.　When　passengers　get　on　or　off　the　bus,　they　often　hide　from　each　other.　It　is　a　great　challenge　for　automatically　accounting　passengers　through　camera.　The　traditionally　video-based　target　detection　algorithm　or　target　tracking　algorithm　is　difficult　to　accurately　count　the　number　of　passenger　on　and　off.　In　this　paper,　the　YOLOv2　algorithm　is　combined　with　the　MIL　tracker　so　as　to　real-time　account　the　number　of　passengers　in　the　bus　surveillance　video.　Experiment　shows　that　the　accuracy　rate　of　bus　passenger　statistics　proposed　in　this　paper　reaches　over　99%,　and　it　proves　that　our　method　has　good　real-time　and　high　accuracy.　©　2020　IEEE.