Large　Language　Models　(LLMs)　exhibit　robust　problem-solving　capabilities　for　diverse　tasks.　However,　most　LLM-based　agents　are　designed　as　specific　task　solvers　with　sophisticated　prompt　engineering,　rather　than　agents　capable　of　learning　and　evolving　through　interactions.　These　task　solvers　necessitate　manually　crafted　prompts　to　inform　task　rules　and　regulate　LLM　behaviors,　inherently　incapacitating　to　address　complex　dynamic　scenarios　e.g.,　large　interactive　games.　In　light　of　this,　we　propose　Agent-Pro:　an　LLM-based　Agent　with　Policy-level　Reflection　and　Optimization　that　can　learn　a　wealth　of　expertise　from　interactive　experiences　and　progressively　elevate　its　behavioral　policy.　Specifically,　it　involves　a　dynamic　belief　generation　and　reflection　process　for　policy　evolution.　Rather　than　action-level　reflection,　Agent-Pro　iteratively　reflects　on　past　trajectories　and　beliefs,　＇fine-tuning＇　its　irrational　beliefs　for　a　better　policy.　Moreover,　a　depth-first　search　is　employed　for　policy　optimization,　ensuring　continual　enhancement　in　policy　payoffs.　Agent-Pro　is　evaluated　across　two　games:　Blackjack　and　Texas　Hold＇em,　outperforming　vanilla　LLM　and　specialized　models.　Our　results　show　Agent-Pro　can　learn　and　evolve　in　complex　and　dynamic　scenes,　which　also　benefits　numerous　LLM-based　applications.　©　2024　Association　for　Computational　Linguistics.