In　this　article,　an　accelerated　Q　-learning　algorithm　with　evolving　control　is　established　to　solve　the　optimal　tracking　control　problem.　First,　an　accelerated　Q　-learning　scheme　is　constructed　with　an　advanced　Q　-function.　By　utilizing　the　advanced　Q　-function,　calculating　of　the　feedforward　control　input　can　be　avoided　and　the　terminal　tracking　error　can　be　eliminated.　Then,　by　introducing　the　relaxation　factor,　the　convergence　rate　of　the　iterative　Q　-function　sequence　is　accelerated　significantly,　which　is　a　potential　way　to　diminish　the　computational　burden.　Furthermore,　the　convergence,　positive　definiteness,　and　stability　conditions　of　the　accelerated　Q　-learning　algorithm　are　analyzed　with　some　preconditions　of　the　relaxation　factor.　Thus,　the　developed　algorithm　can　achieve　evolving　control.　Finally,　the　fantastic　performance　of　the　developed　algorithm　with　critic　network　implementation　is　verified　through　two　simulation　examples.