The　edge　information　plays　a　key　role　in　the　restoration　of　a　depth　map.　Most　conventional　methods　assume　that　the　color　image　and　depth　map　are　consistent　in　edge　areas.　However,　complex　texture　regions　in　the　color　image　do　not　match　exactly　with　edges　in　the　depth　map.　In　this　paper,　firstly,　we　point　out　that　in　most　cases　the　consistency　between　normal　map　and　depth　map　is　much　higher　than　that　between　RGB-D　pairs.　Then　we　propose　a　dual　normal-depth　regularization　term　to　guide　the　restoration　of　depth　map,　which　constrains　the　edge　consistency　between　normal　map　and　depth　map　back　and　forth.　Moreover,　considering　the　bimodal　characteristic　of　weight　distribution　that　exists　in　depth　discontinuous　areas,　a　reweighted　graph　Laplacian　regularizer　is　proposed　to　promote　this　bimodal　characteristic.　And　this　regularization　is　incorporated　into　a　unified　optimization　framework　to　effectively　protect　the　piece-wise　smoothness(PWS)　characteristics　of　depth　map.　By　treating　depth　image　as　graph　signal,　the　weight　between　two　nodes　is　adapted　according　to　its　content.　The　proposed　method　is　tested　for　both　noise-free　and　noisy　cases,　and　is　compared　against　the　state-of-the-art　methods　on　both　synthesis　and　real　captured　datasets.　Extensive　experimental　results　demonstrate　the　superior　performance　of　our　method　compared　with　most　state-of-the-art　works　in　terms　of　both　objective　and　subjective　quality　evaluations.　Specifically,　our　method　is　more　effective　on　edge　areas　and　more　robust　to　noises.