Indexed by:
Abstract:
With complex two-dimensional structure, offline handwritten mathematical expressions is difficult to recognize due to the variable scale of their symbols and the various transformation of their writing styles. This paper proposed a mutual learning model based on multi-scale feature fusion. Firstly, to enhance the model for extracting fine-grained information from expressions and comprehending semantic information of global two-dimensional structures, multi-scale feature fusion was introduced in the encoding stage. Secondly, paired handwritten and printed mathematical expressions were introduced for training the mutual learning model, which includes decoder loss and context matching loss to learn LaTeX grammar as well as semantic invariance between handwritten and printed mathematical expressions respectively to improve the robustness of the model to different writing styles. Experimental validation was performed on the CROHME 2014/2016/2019 dataset. After introducing the multi-scale feature fusion mechanism, the expression correctness rate reaches 55. 25%, 52. 31%, 53. 72%, respectively. After introducing the mutual learning mechanism, the expression correct rate reaches 55. 43%, 53. 53%, 53. 79%, respectively. The expression correctness rate reaches 58. 88%, 55. 10%, 57. 05% after introducing both mechanisms at the same time. It is proved experimentally that the proposed method can effectively extract the features in formulas at different scales and overcome the problems of different handwriting styles and small amount of data by mutual learning mechanism. In addition, the experimental results on the HME100K dataset verified the effectiveness of the proposed model. © 2024 South China University of Technology. All rights reserved.
Keyword:
Reprint Author's Address:
Email:
Source :
Journal of South China University of Technology (Natural Science)
ISSN: 1000-565X
Year: 2024
Issue: 2
Volume: 52
Page: 23-31
Cited Count:
WoS CC Cited Count: 0
SCOPUS Cited Count:
ESI Highly Cited Papers on the List: 0 Unfold All
WanFang Cited Count:
Chinese Cited Count:
30 Days PV: 6
Affiliated Colleges: