• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
搜索

Author:

Gu, Jiajun (Gu, Jiajun.) | Wang, Zhiyong (Wang, Zhiyong.) (Scholars:王智勇) | Ouyang, Wanli (Ouyang, Wanli.) | Zhang, Weichen (Zhang, Weichen.) | Li, Jiafeng (Li, Jiafeng.) | Zhuo, Li (Zhuo, Li.)

Indexed by:

EI Scopus

Abstract:

Estimating 3D hand pose from a single RGB image is a challenging task because of its ill-posed nature (i.e., depth ambiguity). Recently, various generative approaches have been proposed to predict the 3D joints of an RGB hand image by learning a unified latent space between two modalities (i.e., RGB image and 3D joints). However, projecting multi-modal data (i.e., RGB images and 3D joints) into a unified latent space is difficult as the modality-specific features usually interfere the learning of the optimal latent space. Hence in this paper, we propose to disentangle the latent space into two sub-latent spaces: modality- specific latent space and pose-specific latent space for 3D hand pose estimation. Our proposed method, namely Disentangled Cross-Modal Latent Space (DCMLS), consists of two variational autoencoder networks and auxiliary components which connect the two VAEs to align underlying hand poses and transfer modality-specific context from RGB to 3D. For the hand pose latent space, we align it with the two modalities by using a cross-modal discriminator with an adversarial learning strategy. For the context latent space, we learn a context translator to gain access to the cross-modal context. Experimental results on two widely used public benchmark datasets RHD and STB demonstrate that our proposed DCMLS method is able to clearly outperform the state-of-the-art ones on single image based 3D hand pose estimation. © 2020 IEEE.

Keyword:

Computer vision Modal analysis Learning systems

Author Community:

  • [ 1 ] [Gu, Jiajun]University of Sydney, Australia
  • [ 2 ] [Wang, Zhiyong]University of Sydney, Australia
  • [ 3 ] [Ouyang, Wanli]University of Sydney, Australia
  • [ 4 ] [Zhang, Weichen]University of Sydney, Australia
  • [ 5 ] [Li, Jiafeng]Beijing University of Technology, China
  • [ 6 ] [Zhuo, Li]Beijing University of Technology, China

Reprint Author's Address:

Email:

Show more details

Related Keywords:

Related Article:

Source :

Year: 2020

Page: 380-389

Language: English

Cited Count:

WoS CC Cited Count:

SCOPUS Cited Count: 14

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 2

Online/Total:620/10635330
Address:BJUT Library(100 Pingleyuan,Chaoyang District,Beijing 100124, China Post Code:100124) Contact Us:010-67392185
Copyright:BJUT Library Technical Support:Beijing Aegean Software Co., Ltd.