Knowledge　Distillation　transfers　knowledge　learned　by　a　teacher　network　to　a　student　network.　A　common　mode　of　knowledge　transfer　is　directly　using　the　teacher　network’s　experience　for　all　samples　without　differentiating　whether　the　experience　of　teacher　is　successful　or　not.　According　to　common　sense,　experience　varies　with　its　nature.　Successful　experience　is　used　for　guidance,　and　failed　experience　is　used　for　correction.　Inspired　by　that,　this　paper　analyzes　the　failure　of　teacher　and　proposes　a　reflective　learning　paradigm,　which　additional　uses　heuristic　knowledge　extracted　from　the　teacher’s　failure　besides　following　the　authority　of　teacher.　Specifically,　this　paper　defines　Mutual　Error　Distance　(MED)　based　on　the　teacher’s　wrong　predictions.　MED　measures　the　adequacy　of　the　decision　boundary　learned　by　teacher,　which　concretizes　the　failure　of　teacher.　Then,　this　paper　proposes　DCGD　(divide-and-conquer　grouping　distillation)　to　critically　transfer　the　teacher’s　knowledge　by　grouping　the　target　task　into　small-scale　subtasks　and　designing　multi-branch　networks　on　the　basis　of　MED.　Finally,　a　switchable　training　mechanism　is　designed　to　integrate　a　regular　student　which　provides　an　option　of　student　network　without　parameter　addition　compared　with　the　multi-branch　student　network.　Extensive　experiments　on　three　image　classification　benchmarks　(CIFAR-10,　CIFAR-100　and　TinyImageNet)　show　the　effectiveness　of　the　proposed　paradigm.　Especially　on　CIFAR-100　dataset,　the　average　error　of　students　using　DCGD+DKD　decreased　by　4.28%.　In　addition,　the　experiment　results　show　that　the　paradigm　is　also　applicable　to　self-distillation.　IEEE