• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
搜索

Author:

Su, Liying (Su, Liying.) | Wang, Yitao (Wang, Yitao.) | Zhai, Dezhao (Zhai, Dezhao.) | Shi, Yuping (Shi, Yuping.) | Ding, Yinghao (Ding, Yinghao.) | Gao, Guohua (Gao, Guohua.) | Li, Qinwei (Li, Qinwei.) | Yu, Ming (Yu, Ming.) | Wu, Hang (Wu, Hang.)

Indexed by:

EI Scopus SCIE

Abstract:

Heart rate (HR) is an important indicator reflecting the overall physical and mental health of the human body, playing a crucial role in diagnosing cardiovascular and neurological diseases. Recent research has revealed that variations in the light absorption of human skin captured through facial video over the cardiac cycle, due to changes in blood volume, can be utilized for non-contact HR estimation. However, most existing methods rely on single-modal video sources (such as RGB or NIR), which often yield suboptimal results due to noise and the limitations of a single information source. To overcome these challenges, this paper proposes a multimodal information fusion architecture named the spatiotemporal sensitive network (SS-Net) for non-contact heart rate estimation. Firstly, spatiotemporal feature maps are utilized to extract physiological signals from RGB and NIR videos effectively. Next, a spatiotemporal sensitive (SS) module is introduced to extract useful physiological signal information from both RGB and NIR spatiotemporal maps. Finally, a multi-level spatiotemporal context fusion (MLSC) module is designed to fuse and complement information between the visible light and infrared modalities. Then, different levels of fused features are refined in task-specific branches to predict both remote photoplethysmography (rPPG) signals and heart rate (HR) signals. Experiments conducted on three datasets demonstrate that the proposed SS-Net achieves superior performance compared to existing methods.

Keyword:

heart rate multi-level fusion multimodal fusion visible and infrared facial videos spatiotemporal sensitive video Swin transformer

Author Community:

  • [ 1 ] [Su, Liying]Beijing Univ Technol, Coll Mech & Energy Engn, Beijing 100124, Peoples R China
  • [ 2 ] [Wang, Yitao]Beijing Univ Technol, Coll Mech & Energy Engn, Beijing 100124, Peoples R China
  • [ 3 ] [Gao, Guohua]Beijing Univ Technol, Coll Mech & Energy Engn, Beijing 100124, Peoples R China
  • [ 4 ] [Zhai, Dezhao]Tianjin Univ Technol, Sch Mech Engn, Tianjin Key Lab Adv Mechatron Syst Design & Intell, Tianjin 300384, Peoples R China
  • [ 5 ] [Ding, Yinghao]Tianjin Univ Technol, Sch Mech Engn, Tianjin Key Lab Adv Mechatron Syst Design & Intell, Tianjin 300384, Peoples R China
  • [ 6 ] [Zhai, Dezhao]Tianjin Univ Technol, Natl Demonstrat Ctr Expt Mech & Elect Engn Educ, Tianjin 300384, Peoples R China
  • [ 7 ] [Ding, Yinghao]Tianjin Univ Technol, Natl Demonstrat Ctr Expt Mech & Elect Engn Educ, Tianjin 300384, Peoples R China
  • [ 8 ] [Shi, Yuping]Civil Aviat Univ China, Tianjin Key Lab Adv Signal Proc, Tianjin 300300, Peoples R China
  • [ 9 ] [Li, Qinwei]Civil Aviat Univ China, Tianjin Key Lab Adv Signal Proc, Tianjin 300300, Peoples R China
  • [ 10 ] [Yu, Ming]Acad Mil Sci, Syst Engn Inst, Peoples Liberat Army, Tianjin 300161, Peoples R China
  • [ 11 ] [Wu, Hang]Acad Mil Sci, Syst Engn Inst, Peoples Liberat Army, Tianjin 300161, Peoples R China
  • [ 12 ] [Wu, Hang]Nankai Univ, Sch Artificial Intelligence, Tianjin 300381, Peoples R China

Reprint Author's Address:

  • [Li, Qinwei]Civil Aviat Univ China, Tianjin Key Lab Adv Signal Proc, Tianjin 300300, Peoples R China;;[Wu, Hang]Acad Mil Sci, Syst Engn Inst, Peoples Liberat Army, Tianjin 300161, Peoples R China;;[Wu, Hang]Nankai Univ, Sch Artificial Intelligence, Tianjin 300381, Peoples R China;;

Show more details

Related Keywords:

Source :

APPLIED SCIENCES-BASEL

Year: 2024

Issue: 20

Volume: 14

2 . 7 0 0

JCR@2022

Cited Count:

WoS CC Cited Count: 1

SCOPUS Cited Count: 1

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 1

Affiliated Colleges:

Online/Total:132/10623387
Address:BJUT Library(100 Pingleyuan,Chaoyang District,Beijing 100124, China Post Code:100124) Contact Us:010-67392185
Copyright:BJUT Library Technical Support:Beijing Aegean Software Co., Ltd.