Simple　Summary　Breast　cancer　is　leading　cancer　increases　the　death　rate　in　women.　Early　diagnosis　of　breast　cancer　in　women　can　save　their　lives.　The　current　study　proposed　a　novel　scheme　to　detect　architectural　distortion　from　mammogram　images　to　predict　breast　cancer　using　a　deep　learning　approach.　Results　are　evaluated　on　a　public　and　a　private　dataset　which　may　help　to　improve　the　diagnostic　ability　of　breast　cancer　of　radiologists　and　doctors　in　daily　clinical　routines.　Furthermore,　the　proposed　method　achieved　maximum　accuracy　as　compared　with　previous　approaches.　This　study　can　be　interesting　and　valuable　in　the　healthcare　predictive　modeling　domain　and　will　add　a　real　contribution　to　society.　Architectural　distortion　is　the　third　most　suspicious　appearance　on　a　mammogram　representing　abnormal　regions.　Architectural　distortion　(AD)　detection　from　mammograms　is　challenging　due　to　its　subtle　and　varying　asymmetry　on　breast　mass　and　small　size.　Automatic　detection　of　abnormal　ADs　regions　in　mammograms　using　computer　algorithms　at　initial　stages　could　help　radiologists　and　doctors.　The　architectural　distortion　star　shapes　ROIs　detection,　noise　removal,　and　object　location,　affecting　the　classification　performance,　reducing　accuracy.　The　computer　vision-based　technique　automatically　removes　the　noise　and　detects　the　location　of　objects　from　varying　patterns.　The　current　study　investigated　the　gap　to　detect　architectural　distortion　ROIs　(region　of　interest)　from　mammograms　using　computer　vision　techniques.　Proposed　an　automated　computer-aided　diagnostic　system　based　on　architectural　distortion　using　computer　vision　and　deep　learning　to　predict　breast　cancer　from　digital　mammograms.　The　proposed　mammogram　classification　framework　pertains　to　four　steps　such　as　image　preprocessing,　augmentation　and　image　pixel-wise　segmentation.　Architectural　distortion　ROI＇s　detection,　training　deep　learning,　and　machine　learning　networks　to　classify　AD＇s　ROIs　into　malignant　and　benign　classes.　The　proposed　method　has　been　evaluated　on　three　databases,　the　PINUM,　the　CBIS-DDSM,　and　the　DDSM　mammogram　images,　using　computer　vision　and　depth-wise　2D　V-net　64　convolutional　neural　networks　and　achieved　0.95,　0.97,　and　0.98　accuracies,　respectively.　Experimental　results　reveal　that　our　proposed　method　outperforms　as　compared　with　the　ShuffelNet,　MobileNet,　SVM,　K-NN,　RF,　and　previous　studies.