• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
搜索

Author:

Gu, Jiajun (Gu, Jiajun.) | Wang, Zhiyong (Wang, Zhiyong.) (Scholars:王智勇) | Ouyang, Wanli (Ouyang, Wanli.) | Zhang, Weichen (Zhang, Weichen.) | Li, Jiafeng (Li, Jiafeng.) | Zhuo, Li (Zhuo, Li.) (Scholars:卓力)

Indexed by:

CPCI-S

Abstract:

Estimating 3D hand pose from a single RGB image is a challenging task because of its ill-posed nature (i.e., depth ambiguity). Recently, various generative approaches have been proposed to predict the 3D joints of an RGB hand image by learning a unified latent space between two modalities (i.e., RGB image and 3D joints). However, projecting multi-modal data (i.e., RGB images and 3D joints) into a unified latent space is difficult as the modality-specific features usually interfere the learning of the optimal latent space. Hence in this paper, we propose to disentangle the latent space into two sub-latent spaces: modalityspecific latent space and pose-specific latent space for 3D hand pose estimation. Our proposed method, namely Dis-entangled Cross-Modal Latent Space (DCMLS), consists of two variational autoencoder networks and auxiliary components which connect the two VAEs to align underlying hand poses and transfer modality-specific context from RGB to 3D. For the hand pose latent space, we align it with the two modalities by using a cross-modal discriminator with an adversarial learning strategy. For the context latent space, we learn a context translator to gain access to the cross-modal context. Experimental results on two widely used public benchmark datasets RHD and STB demonstrate that our proposed DCMLS method is able to clearly out-perform the state-of-the-art ones on single image based 3D hand pose estimation.

Keyword:

Author Community:

  • [ 1 ] [Gu, Jiajun]Univ Sydney, Sydney, NSW, Australia
  • [ 2 ] [Wang, Zhiyong]Univ Sydney, Sydney, NSW, Australia
  • [ 3 ] [Ouyang, Wanli]Univ Sydney, Sydney, NSW, Australia
  • [ 4 ] [Zhang, Weichen]Univ Sydney, Sydney, NSW, Australia
  • [ 5 ] [Li, Jiafeng]Beijing Univ Technol, Beijing, Peoples R China
  • [ 6 ] [Zhuo, Li]Beijing Univ Technol, Beijing, Peoples R China

Reprint Author's Address:

  • [Gu, Jiajun]Univ Sydney, Sydney, NSW, Australia

Show more details

Related Keywords:

Related Article:

Source :

2020 IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV)

ISSN: 2472-6737

Year: 2020

Page: 380-389

Language: English

Cited Count:

WoS CC Cited Count: 7

SCOPUS Cited Count:

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 5

Online/Total:605/10706878
Address:BJUT Library(100 Pingleyuan,Chaoyang District,Beijing 100124, China Post Code:100124) Contact Us:010-67392185
Copyright:BJUT Library Technical Support:Beijing Aegean Software Co., Ltd.