Annotating　image　regions　has　been　a　challenging　open　issue　in　many　areas　such　as　image　content　understanding　and　image　retrieval.　In　this　paper,　rather　than　solely　rely　on　visual　features　of　image　regions,　a　novel　approach　is　proposed　to　improve　region　annotation　by　taking　concept　constraints　into　account,　since　high　level　conceptual　information　such　as　image　categories　can　increase　the　confidence　of　possible　region　labels　as　well　as　decrease　the　confidence　of　impossible　region　labels.　We　employ　statistical　models　to　learn　the　relationships　among　visual　features,　image　concepts,　and　region　labels.　As　a　result,　a　set　of　possible　region　labels　can　be　derived　from　a　set　of　visual　feature　vectors　of　a　given　image　so　as　to　refine　the　annotation　output　obtained　by　using　visual　feature　only.　Promising　experimental　results　have　been　demonstrated　on　8462　regions　of　the　University　of　Washington　image　dataset　with　diverse　concepts　for　the　proposed　approach.　©　2007　IEEE.