In　this　article,　an　advanced　accelerated　Q-learning　(AQL)　approach　is　designed　to　address　the　nonlinear　discrete-time　optimal　tracking　problem　of　zero-sum　games　with　unknown　dynamics.　Different　from　conventional　adaptive　dynamic　programming　methods,　the　advanced　Q-learning　algorithm　incorporates　both　the　control　input　and　the　disturbance　signal　into　the　tracking　error,　which　obviates　the　quadratic　form　of　control　and　disturbance　inputs　directly.　This　innovative　Q-function　is　used　to　derive　the　optimal　tracking　control　policy　pair　that　ensures　the　terminal　tracking　error　asymptotically　converges　to　zero,　independent　of　the　feedforward　control　input.　In　order　to　improve　the　convergence　speed　of　the　iterative　process　and　reduce　computational　complexity,　an　accelerated　factor　is　introduced.　After　collecting　offline　input–output　data,　a　backpropagation　neural　network　is　employed　to　approximate　the　proposed　Q-function,　which　enables　model-free　tracking　control　of　zero-sum　games　through　an　off-policy　learning　mechanism.　Furthermore,　the　theoretical　properties　of　the　developed　algorithm　are　analyzed　under　specific　preconditions.　Finally,　the　effectiveness　of　the　AQL　algorithm　is　validated　through　a　numerical　simulation,　which　is　implemented　using　a　critic-only　structure.　©　The　Author(s),　under　exclusive　licence　to　Springer　Nature　B.V.　2025.