In　this　article,　an　evolution-guided　value　iteration　(EGVI)　algorithm　is　established　to　address　optimal　tracking　problems　for　nonlinear　nonaffine　systems.　Conventional　adaptive　dynamic　programming　algorithms　rely　on　gradient　information　to　improve　the　policy,　which　adheres　to　the　first　order　necessity　condition.　Nonetheless,　these　methods　encounter　limitations　when　gradient　information　is　intricate　or　system　dynamics　lack　differentiability.　In　response　to　this　challenge,　evolutionary　computation　is　leveraged　by　EGVI　to　search　for　the　optimal　policy　without　requiring　an　action　network.　The　competition　within　the　policy　population　serves　as　the　driving　force　for　policy　improvement.　Therefore,　EGVI　can　effectively　handle　complex　and　non-differentiable　systems.　Additionally,　this　innovative　method　has　the　potential　to　enhance　exploration　efficiency　and　bolster　the　robustness　of　algorithms　due　to　its　population-based　characteristics.　Furthermore,　the　convergence　of　the　algorithm　and　the　stability　of　the　policy　are　investigated　based　on　the　EGVI　framework.　Finally,　the　effectiveness　of　the　established　method　is　comprehensively　demonstrated　through　two　simulation　experiments.　©　2024　Elsevier　B.V.