In　recent　years,　the　application　of　function　approximators,　such　as　neural　networks　and　polynomials,　has　ushered　in　a　new　stage　of　development　in　solving　optimal　control　problems.　However,　considering　the　existence　of　approximation　errors,　the　stability　of　the　controlled　system　cannot　be　guaranteed.　Therefore,　in　view　of　the　prevalence　of　approximation　errors,　we　investigate　optimal　tracking　control　problems　for　discrete-time　systems.　First,　a　novel　value　function　is　introduced　into　the　intelligent　critic　framework.　Second,　an　implicit　method　is　utilized　to　demonstrate　the　boundedness　of　the　iterative　value　functions　with　approximation　errors.　An　explicit　method　is　applied　to　prove　the　stability　of　the　system　with　approximation　errors.　Furthermore,　an　evolving　policy　is　designed　to　iteratively　tackle　the　optimal　tracking　control　problem　and　demonstrate　the　stability　of　the　system.　Finally,　the　effectiveness　of　the　developed　method　is　verified　through　numerical　as　well　as　practical　examples.