• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
搜索

Author:

Cheng, Hao (Cheng, Hao.) | Cao, Jiahang (Cao, Jiahang.) | Xiao, Erjia (Xiao, Erjia.) | Sun, Mengshu (Sun, Mengshu.) | Xu, Renjing (Xu, Renjing.)

Indexed by:

EI

Abstract:

Deploying energy-efficient deep learning algorithms on computational-limited devices, such as robots, is still a pressing issue for real-world applications. Spiking Neural Networks (SNNs), a novel brain-inspired algorithm, offer a promising solution due to their low-latency and low-energy properties over traditional Artificial Neural Networks (ANNs). Despite their advantages, the dense structure of deep SNNs can still result in extra energy consumption. The Lottery Ticket Hypothesis (LTH) posits that within dense neural networks, there exist winning Lottery Tickets (LTs), namely sub-networks, that can be obtained without compromising performance. Inspired by this, this paper delves into the spiking-based LTs (SLTs), examining their unique properties and potential for extreme efficiency. Then, two significant sparse Rewards are gained through comprehensive explorations and meticulous experiments on SLTs across various dense structures. Moreover, a sparse algorithm tailored for spiking transformer structure, which incorporates convolution operations into the Patch Embedding Projection (ConvPEP) module, has been proposed to achieve Multi-level Sparsity (MultiSp). MultiSp refers to (1) Patch number sparsity; (2) ConvPEP weights sparsity and binarization; and (3) ConvPEP activation layer binarization. Extensive experiments demonstrate that our method achieves extreme sparsity with only a slight performance decrease, paving the way for deploying energy-efficient neural networks in robotics and beyond. © 2024 IEEE.

Keyword:

Robot applications Robot learning Multilayer neural networks Deep neural networks Energy utilization

Author Community:

  • [ 1 ] [Cheng, Hao]The Hong Kong University of Science and Technology (Guangzhou), Mics Thrust, Hong Kong
  • [ 2 ] [Cao, Jiahang]The Hong Kong University of Science and Technology (Guangzhou), Mics Thrust, Hong Kong
  • [ 3 ] [Xiao, Erjia]The Hong Kong University of Science and Technology (Guangzhou), Mics Thrust, Hong Kong
  • [ 4 ] [Sun, Mengshu]Beijing University of Technology, China
  • [ 5 ] [Xu, Renjing]The Hong Kong University of Science and Technology (Guangzhou), Mics Thrust, Hong Kong

Reprint Author's Address:

Email:

Show more details

Related Keywords:

Related Article:

Source :

ISSN: 2153-0858

Year: 2024

Page: 442-449

Language: English

Cited Count:

WoS CC Cited Count:

SCOPUS Cited Count:

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 10

Affiliated Colleges:

Online/Total:607/10572661
Address:BJUT Library(100 Pingleyuan,Chaoyang District,Beijing 100124, China Post Code:100124) Contact Us:010-67392185
Copyright:BJUT Library Technical Support:Beijing Aegean Software Co., Ltd.