Deep　learning　have　achieved　great　success　both　in　image　and　natural　language　processing.　When　to　search　similar　images,　there　are　some　data　occur　which　image　and　image　title　are　not　related.　To　deal　with　this　problem　which　involves　the　process　of　both　image　and　natural　language,　we　propose　a　convolutional　neural　network　model.　The　model　both　uses　the　feature　of　images　and　texts　to　judge　the　similarity.　In　the　model,　the　two　type　of　feature　extracted　respectively　and　then　give　the　probability　of　the　relationship　between　images　and　titles.　This　probability　is　added　to　the　search　strategy　as　a　score　to　improve　search　quality.　©　2018　IEEE.