• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
搜索

Author:

Liu, W. (Liu, W..) | Jia, K. (Jia, K..) | Sun, Z. (Sun, Z..) | Feng, J. (Feng, J..) | Zhao, X. (Zhao, X..)

Indexed by:

Scopus

Abstract:

Disease analysis using multimodal physiological signals is currently a hot research area. Aiming at the problem that current multimodal feature fusion approaches neglect the correlation between different modalities, this paper proposes a hybrid multimodal fusion model based on the attention mechanism for depression recognition. The model selects two modalities, EEG and speech signals, for feature fusion. In the feature extraction stage of EEG signals, the correlation between features is analyzed using Spearman’s rank correlation coefficient to achieve dimensionality reduction. In the feature-level modal fusion stage, the attention mechanism is applied to automatically learn the contribution of different features and the complementarity between different modalities. In the model fusion layer, a multi-layer long and short-term memory network (ML-LSTM) is adopted, which makes full use of the complementarity between the features of EEG signals and audio signals. Finally, the outputs of feature-level fusion and model-level fusion are weighted and combined to obtain the final prediction results. Experimental results on the MODMA dataset show that the accuracy of the proposed multimodal fusion model for depression recognition reached 88.54%. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

Keyword:

Author Community:

  • [ 1 ] [Liu W.]Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China
  • [ 2 ] [Liu W.]Beijing Laboratory of Advanced Information Networks, Beijing, 100124, China
  • [ 3 ] [Liu W.]Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing, 100124, China
  • [ 4 ] [Jia K.]Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China
  • [ 5 ] [Jia K.]Beijing Laboratory of Advanced Information Networks, Beijing, 100124, China
  • [ 6 ] [Jia K.]Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing, 100124, China
  • [ 7 ] [Sun Z.]Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China
  • [ 8 ] [Sun Z.]Beijing Laboratory of Advanced Information Networks, Beijing, 100124, China
  • [ 9 ] [Sun Z.]Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing, 100124, China
  • [ 10 ] [Feng J.]Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China
  • [ 11 ] [Feng J.]Beijing Laboratory of Advanced Information Networks, Beijing, 100124, China
  • [ 12 ] [Feng J.]Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing, 100124, China
  • [ 13 ] [Zhao X.]Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China
  • [ 14 ] [Zhao X.]Beijing Laboratory of Advanced Information Networks, Beijing, 100124, China
  • [ 15 ] [Zhao X.]Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing, 100124, China

Reprint Author's Address:

Email:

Show more details

Related Keywords:

Related Article:

Source :

ISSN: 0302-9743

Year: 2025

Volume: 15541 LNAI

Page: 194-203

Language: English

Cited Count:

WoS CC Cited Count:

SCOPUS Cited Count:

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 3

Affiliated Colleges:

Online/Total:735/10503981
Address:BJUT Library(100 Pingleyuan,Chaoyang District,Beijing 100124, China Post Code:100124) Contact Us:010-67392185
Copyright:BJUT Library Technical Support:Beijing Aegean Software Co., Ltd.