Optimal　control　problems　are　ubiquitous　in　practical　engineering　applications　and　social　life　with　the　idea　of　cost　or　resource　conservation.　Based　on　the　critic　learning　scheme,　adaptive　dynamic　programming　(ADP)　is　regarded　as　a　significant　avenue　to　address　the　optimal　control　problems　by　combining　the　advanced　design　ideas　such　as　adaptive　control,　reinforcement　learning,　and　intelligent　control.　This　survey　introduces　the　recent　development　of　ADP　and　related　intelligent　critic　control　with　an　emphasis　on　advanced　value　iteration　(VI)　schemes　for　discrete-time　nonlinear　systems.　The　theoretical　results　focus　on　convergence　and　stability　properties　for　general　VI,　stabilizing　VI,　integrated　VI,　evolving　VI,　adjustable　VI　schemes　and　so　on.　Several　significant　applications　are　also　elaborated　in　aspects　of　optimal　regulation,　optimal　tracking,　and　zero-sum　games.　We　aim　to　break　through　the　bottleneck　problems　for　VI　algorithms　in　realizing　evolving　control,　accelerating　learning　speed,　and　reducing　the　calculation　expense.　In　addition,　the　prospects　of　new　theoretical　and　technical　fields　for　advanced　VI　schemes　are　looked　ahead.