Wafer　map　inspection　is　essential　for　semiconductor　manufacturing　quality　control　and　analysis.　The　deep　convolutional　neural　network　(DCNN)　is　the　most　effective　algorithm　in　wafer　defect　pattern　analysis.　Traditional　DCNNs　rely　heavily　on　high　quality　datasets　for　training.　However,　obtaining　balanced　and　sufficient　labeled　data　is　difficult　in　practice.　This　paper　reconsiders　the　causes　of　the　imbalance　and　proposes　a　deep　learning　method　that　can　learn　robust　knowledge　from　an　imbalanced　dataset　using　the　attention　mechanism　and　cosine　normalization.　We　interpret　the　dataset　imbalance　as　both　a　feature　and　a　quantity　distribution　imbalance.　To　compensate　for　feature　distribution　imbalance,　we　add　an　improved　convolutional　attention　module　to　the　DCNN　to　enhance　representation.　In　particular,　a　feature-map-specific　direction　mapping　module　is　developed　to　amplify　the　positional　information　of　defect　clusters.　For　quantity　distribution　imbalance,　the　cosine　normalization　algorithm　is　proposed　to　replace　the　fully　connected　layer,　and　classifier　fine-tuning　is　realized　through　a　small　amount　of　iterative　training,　which　decreases　the　sensitivity　to　the　quantitative　distribution.　The　experimental　results　on　real-world　datasets　demonstrate　that　the　proposed　method　significantly　improves　the　robustness　of　wafer　map　inspection　and　outperforms　existing　algorithms　when　trained　on　imbalanced　datasets.