• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
搜索

Author:

Zhang, Xiaodan (Zhang, Xiaodan.) | Yang, Sisi (Yang, Sisi.) | Shi, Yanzhao (Shi, Yanzhao.) | Ji, Junzhong (Ji, Junzhong.) | Liu, Ying (Liu, Ying.) | Wang, Zheng (Wang, Zheng.) | Xu, Huimin (Xu, Huimin.)

Indexed by:

EI Scopus SCIE

Abstract:

Brain Computed Tomography (CT) report generation, which aims to assist radiologists in diagnosing cerebrovascular diseases efficiently, is challenging in feature representation for dozens of images and language descriptions with several sentences. Existing report generation methods have achieved significant achievement based on the encoder-decoder framework and attention mechanism. However, current research has limitations in solving the many-to-many alignment between the multi-images of Brain CT imaging and the multi-sentences of Brain CT report, and fails to attend to critical images and lesion areas, resulting in inaccurate descriptions. In this paper, we propose a novel Weakly Guided Attention Model with Hierarchical Interaction, named WGAM-HI, to improve Brain CT report generation. Specifically, WGAM-HI conducts many-to-many matching for multiple visual images and semantic sentences via a hierarchical interaction framework with a two -layer attention model and a two-layer report generator. In addition, two weakly guided mechanisms are proposed to facilitate the attention model to focus more on important images and lesion areas under the guidance of pathological events and Gradient-weighted Class Activation Mapping (Grad-CAM) respectively. The pathological event acts as a bridge between the essential serial images and the corresponding sentence, and the Grad-CAM bridges the lesion areas and pathology words. Therefore, under the hierarchical interaction with the weakly guided attention model, the report generator generates more accurate words and sentences. Experiments on the Brain CT dataset demonstrate the effectiveness of WGAM-HI in attending to important images and lesion areas gradually, and generating more accurate reports.

Keyword:

Brain CT Hierarchical interaction Weakly guided attention Medical report generation

Author Community:

  • [ 1 ] [Zhang, Xiaodan]Beijing Univ Technol, Fac Informat Technol, Beijing, Peoples R China
  • [ 2 ] [Yang, Sisi]Beijing Univ Technol, Fac Informat Technol, Beijing, Peoples R China
  • [ 3 ] [Shi, Yanzhao]Beijing Univ Technol, Fac Informat Technol, Beijing, Peoples R China
  • [ 4 ] [Ji, Junzhong]Beijing Univ Technol, Fac Informat Technol, Beijing, Peoples R China
  • [ 5 ] [Liu, Ying]Peking Univ Third Hosp, Dept Radiol, Beijing, Peoples R China
  • [ 6 ] [Wang, Zheng]Peking Univ Third Hosp, Dept Radiol, Beijing, Peoples R China
  • [ 7 ] [Xu, Huimin]Peking Univ Third Hosp, Dept Radiol, Beijing, Peoples R China

Reprint Author's Address:

Show more details

Related Keywords:

Related Article:

Source :

COMPUTERS IN BIOLOGY AND MEDICINE

ISSN: 0010-4825

Year: 2023

Volume: 167

7 . 7 0 0

JCR@2022

Cited Count:

WoS CC Cited Count:

SCOPUS Cited Count:

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 0

Affiliated Colleges:

Online/Total:1743/10944087
Address:BJUT Library(100 Pingleyuan,Chaoyang District,Beijing 100124, China Post Code:100124) Contact Us:010-67392185
Copyright:BJUT Library Technical Support:Beijing Aegean Software Co., Ltd.