This　paper　leverages　a　value-iteration-based　Q-learning　(VIQL)　scheme　to　tackle　optimal　tracking　problems　for　nonlinear　nonaffine　systems.　The　optimal　policy　is　learned　from　measured　data　instead　of　a　precise　mathematical　model.　Furthermore,　a　novel　criterion　is　proposed　to　determine　the　stability　of　the　iterative　policy　based　on　measured　data.　The　evolving　control　algorithm　is　developed　to　verify　the　proposed　criterion　by　employing　these　stable　policies　for　system　control.　The　advantage　of　the　early　elimination　of　tracking　errors　is　provided　by　this　approach　since　various　stable　policies　can　be　employed　before　obtaining　the　optimal　strategy.　Finally,　the　effectiveness　of　the　developed　algorithm　is　demonstrated　by　a　simulation　experiment.　IEEE