Indexed by:
Abstract:
Disease analysis using multimodal physiological signals is currently a hot research area. Aiming at the problem that current multimodal feature fusion approaches neglect the correlation between different modalities, this paper proposes a hybrid multimodal fusion model based on the attention mechanism for depression recognition. The model selects two modalities, EEG and speech signals, for feature fusion. In the feature extraction stage of EEG signals, the correlation between features is analyzed using Spearman’s rank correlation coefficient to achieve dimensionality reduction. In the feature-level modal fusion stage, the attention mechanism is applied to automatically learn the contribution of different features and the complementarity between different modalities. In the model fusion layer, a multi-layer long and short-term memory network (ML-LSTM) is adopted, which makes full use of the complementarity between the features of EEG signals and audio signals. Finally, the outputs of feature-level fusion and model-level fusion are weighted and combined to obtain the final prediction results. Experimental results on the MODMA dataset show that the accuracy of the proposed multimodal fusion model for depression recognition reached 88.54%. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.
Keyword:
Reprint Author's Address:
Email:
Source :
ISSN: 0302-9743
Year: 2025
Volume: 15541 LNAI
Page: 194-203
Language: English
Cited Count:
SCOPUS Cited Count:
ESI Highly Cited Papers on the List: 0 Unfold All
WanFang Cited Count:
Chinese Cited Count:
30 Days PV: 3
Affiliated Colleges: