• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
搜索

Author:

Xu, K. (Xu, K..) | Wang, L. (Wang, L..) | Zhang, H. (Zhang, H..) | Yin, B. (Yin, B..)

Indexed by:

CPCI-S EI Scopus

Abstract:

Self-knowledge distillation does not require a pre-trained teacher network like traditional knowledge distillation. Existing methods either require additional parameters or require additional memory consumption. To alleviate this problem, this paper proposes a more efficient self-knowledge distillation method, named LRMS (learning from role-model samples). In every mini-batch, LRMS selects out a role-model sample for each sampled category, and takes its prediction as the proxy semantic for the corresponding category. Then, predictions of the other samples are constrained to be consistent with the proxy semantics, which makes the distribution of predictions for samples within the same category more compact. Meanwhile, the regularization targets corresponding to proxy semantics are set with a higher distillation temperature to better utilize the classificatory information about the categories. Experimental results show that diverse architectures achieve improvements on four image classification datasets by using LRMS. Code is acaliable: https://github.com/KAI1179/LRMS  © 2024 IEEE.

Keyword:

Model Compression Neural Networks Image Classification Self-knowledge Distillation

Author Community:

  • [ 1 ] [Xu K.]Beijing University of Technology, Faculty of Information Technology, Beijing, China
  • [ 2 ] [Wang L.]Beijing University of Technology, Faculty of Information Technology, Beijing, China
  • [ 3 ] [Zhang H.]Beijing University of Technology, Faculty of Information Technology, Beijing, China
  • [ 4 ] [Yin B.]Beijing University of Technology, Faculty of Information Technology, Beijing, China

Reprint Author's Address:

Email:

Show more details

Related Keywords:

Source :

ISSN: 1520-6149

Year: 2024

Page: 5185-5189

Language: English

Cited Count:

WoS CC Cited Count:

SCOPUS Cited Count: 1

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 13

Affiliated Colleges:

Online/Total:477/10587030
Address:BJUT Library(100 Pingleyuan,Chaoyang District,Beijing 100124, China Post Code:100124) Contact Us:010-67392185
Copyright:BJUT Library Technical Support:Beijing Aegean Software Co., Ltd.