The　customized　bus　plays　an　important　role　in　catering　the　special　needs　of　bus　travel.　The　optimization　method　of　customized　bus　route　planning　was　studied　considering　the　time　windows　of　passengers.　The　arrival　time　of　customized　bus　at　stops　were　converted　to　equivalent　bus　travel　distances.　Taking　the　minimum　weighted　cost　of　the　bus　operator,　passengers　travel,　and　exhaust　pollution,　a　customized　bus　route　planning　model　based　on　the　time　windows　of　passengers　was　developed.　The　ant　colony　algorithm　was　designed　to　obtain　a　feasible　solution　of　the　planning　model.　Then,　Q-learning　algorithm　based　on　action　model　approximation　was　designed　to　optimize　the　feasible　solution.　The　environment　model　was　mapped　into　a　two-dimensional　raster　model,　the　action　space　was　set,　and　the　reward　and　punishment　functions　were　designed.　The　results　show　that　the　optimized　bus　line　can　screen　out　the　service　stops,　and　the　comprehensive　cost　is　reduced　by　20.2%.　©　ASCE.