Deep　learning　has　made　a　great　deal　of　success　in　processing　images,　audios,　and　natural　languages.　With　the　growth　of　deep　learning,　how　to　describe　deep　neural　networks　unifiedly　is　becoming　an　important　issue.　We　first　formalize　neural　networks　mathematically　with　their　directed　graph　representations,　and　prove　a　generation　theorem　about　the　induced　networks　of　connected　directed　acyclic　graphs.　Then,　we　set　up　a　unified　framework　for　deep　learning　with　capsule　networks.　This　capsule　framework　could　simplify　the　description　of　existing　deep　neural　networks,　and　provide　a　theoretical　basis　of　graphic　designing　and　programming　techniques　for　deep　learning　models,　thus　would　be　of　great　significance　to　the　advancement　of　deep　learning.