• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
搜索

Author:

Zheng, Yihao (Zheng, Yihao.) | Luo, Kunming (Luo, Kunming.) | Liu, Shuaicheng (Liu, Shuaicheng.) | Li, Zun (Li, Zun.) | Xiang, Ye (Xiang, Ye.) | Wu, Lifang (Wu, Lifang.) | Zeng, Bing (Zeng, Bing.) | Chen, Chang Wen (Chen, Chang Wen.)

Indexed by:

EI Scopus SCIE

Abstract:

Motions in videos are typically a mixture of local dynamic object motions and global camera motion, which are inconsistent in some cases, and even interfere with each other, causing difficulties in various downstream applications, such as video stabilization that requires the global motion, and action recognition that consumes local motions. Therefore, it is crucial to estimate them separately. Existing methods separate two motions from the mixed motion fields, such as optical flow. However, the quality of mixed motion determines the higher bounds of the performance. In this work, we propose a framework, GLOCAL, to directly estimate global and local motions simultaneously from adjacent frames in a self-supervised manner. Our GLOCAL consists of a Global Motion Estimation (GME) module and a Local Motion Estimation (LME) module. The GME module involves a mixed motion estimation backbone, an implicit bottleneck structure for feature dimension reduction, and an explicit bottleneck for global motion recovery based on the global motion bases with foreground mask under the training guidance of proposed global reconstruction loss. An attention U-Net is adopted for LME which produces local motions while excluding motion of irrelevant regions under the guidance of proposed local reconstruction loss. Our method can achieve better performance than the existing methods on the homography estimation dataset DHE and the action recognition dataset NCAA and UCF-101.

Keyword:

motion estimation motion pattern Video understanding optical flow

Author Community:

  • [ 1 ] [Zheng, Yihao]Beijing Univ Technol, Fac Informat Technol, Beijing 100124, Peoples R China
  • [ 2 ] [Li, Zun]Beijing Univ Technol, Fac Informat Technol, Beijing 100124, Peoples R China
  • [ 3 ] [Xiang, Ye]Beijing Univ Technol, Fac Informat Technol, Beijing 100124, Peoples R China
  • [ 4 ] [Wu, Lifang]Beijing Univ Technol, Fac Informat Technol, Beijing 100124, Peoples R China
  • [ 5 ] [Luo, Kunming]Megvii Technol, Beijing 100190, Peoples R China
  • [ 6 ] [Liu, Shuaicheng]Univ Elect Sci & Technol China, Sch Informat & Commun Engn, Chengdu 611731, Peoples R China
  • [ 7 ] [Zeng, Bing]Univ Elect Sci & Technol China, Sch Informat & Commun Engn, Chengdu 611731, Peoples R China
  • [ 8 ] [Chen, Chang Wen]Hong Kong Polytech Univ, Dept Comp, Hong Kong 999077, Peoples R China

Reprint Author's Address:

  • [Wu, Lifang]Beijing Univ Technol, Fac Informat Technol, Beijing 100124, Peoples R China

Show more details

Related Keywords:

Related Article:

Source :

PATTERN RECOGNITION LETTERS

ISSN: 0167-8655

Year: 2024

Volume: 178

Page: 91-97

5 . 1 0 0

JCR@2022

Cited Count:

WoS CC Cited Count:

SCOPUS Cited Count:

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 2

Affiliated Colleges:

Online/Total:875/11103508
Address:BJUT Library(100 Pingleyuan,Chaoyang District,Beijing 100124, China Post Code:100124) Contact Us:010-67392185
Copyright:BJUT Library Technical Support:Beijing Aegean Software Co., Ltd.