Glaucoma　is　one　of　the　prevalent　causes　of　blindness　in　the　modern　world.　It　is　a　salient　chronic　eye　disease　that　leads　to　irreversible　vision　loss.　The　impediments　of　glaucoma　can　be　restricted　if　it　is　identified　at　primary　stages.　In　this　paper,　a　novel　two-phase　Optic　Disk　localization　and　Glaucoma　Diagnosis　Network　(ODGNet)　has　been　proposed.　In　the　first　phase,　a　visual　saliency　map　incorporated　with　shallow　CNN　is　used　for　effective　OD　localization　from　the　fundus　images.　In　the　second　phase,　the　transfer　learning-based　pre-trained　models　are　used　for　glaucoma　diagnosis.　The　transfer　learning-based　models　such　as　AlexNet,　ResNet,　and　VGGNet　incorporated　with　saliency　maps　are　evaluated　on　five　public　retinal　datasets　(ORIGA,　HRF,　DRIONS-DB,　DR-HAGIS,　and　RIM-ONE)　to　differentiate　between　normal　and　glaucomatous　images.　This　study＇s　experimental　results　demonstrate　that　the　proposed　ODGNet　evaluated　on　ORIGA　for　glaucoma　diagnosis　is　the　most　predictive　model　and　achieve　95.75,　94.90,　94.75,　and　97.85%　of　accuracy,　specificity,　sensitivity,　and　area　under　the　curve,　respectively.　These　results　indicate　that　the　proposed　OD　localization　method　based　on　the　saliency　map　and　shallow　CNN　is　robust,　accurate　and　saves　the　computational　cost.