The　success　of　retrieval-augmented　language　models　in　various　natural　language　processing　(NLP)　tasks　has　been　constrained　in　automatic　speech　recognition　(ASR)　applications　due　to　challenges　in　constructing　fine-grained　audio-text　datastores.　This　paper　presents　kNN-CTC,　a　novel　approach　that　overcomes　these　challenges　by　leveraging　Connectionist　Temporal　Classification　(CTC)　pseudo　labels　to　establish　frame-level　audio-text　key-value　pairs,　circumventing　the　need　for　precise　ground　truth　alignments.　We　further　introduce　a　“skip-blank”　strategy,　which　strategically　ignores　CTC　blank　frames,　to　reduce　datastore　size.　By　incorporating　a　k-nearest　neighbors　retrieval　mechanism　into　pretrained　CTC　ASR　systems　and　leveraging　a　fine-grained,　pruned　datastore,　kNN-CTC　consistently　achieves　substantial　improvements　in　performance　under　various　experimental　settings.　Our　code　is　available　at　https://github.com/NKUHLT/KNN-CTC.　©　2024　IEEE.