The　bag　of　visual　words　model　(BoW)　and　its　variants　have　demonstrate　their　effectiveness　for　visual　applications　and　have　been　widely　used　by　researchers.　The　BoW　model　first　extracts　local　features　and　generates　the　corresponding　codebook,　the　elements　of　a　codebook　are　viewed　as　visual　words.　The　local　features　within　each　image　are　then　encoded　to　get　the　final　histogram　representation.　However,　the　codebook　is　dataset　dependent　and　has　to　be　generated　for　each　image　dataset.　This　costs　a　lot　of　computational　time　and　weakens　the　generalization　power　of　the　BoW　model.　To　solve　these　problems,　in　this　paper,　we　propose　to　undo　the　dataset　bias　by　codebook　linear　transformation.　To　represent　every　points　within　the　local　feature　space　using　Euclidean　distance,　the　number　of　bases　should　be　no　less　than　the　space　dimensions.　Hence,　each　codebook　can　be　viewed　as　a　linear　transformation　of　these　bases.　In　this　way,　we　can　transform　the　pre-learned　codebooks　for　a　new　dataset.　However,　not　all　of　the　visual　words　are　equally　important　for　the　new　dataset,　it　would　be　more　effective　if　we　can　make　some　selection　using　sparsity　constraints　and　choose　the　most　discriminative　visual　words　for　transformation.　We　propose　an　alternative　optimization　algorithm　to　jointly　search　for　the　optimal　linear　transformation　matrixes　and　the　encoding　parameters.　Image　classification　experimental　results　on　several　image　datasets　show　the　effectiveness　of　the　proposed　method.　Copyright　©　2013　ACM.