Diverse 3D Hand Gesture Prediction from Body Dynamics by Bilateral Hand Disentanglement
- Publisher:
- IEEE
- Publication Type:
- Conference Proceeding
- Citation:
- 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, 00, pp. 4616-4626
- Issue Date:
- 2023
Closed Access
| Filename | Description | Size | |||
|---|---|---|---|---|---|
| 1666962.pdf | Published version | 844.59 kB |
Copyright Clearance Process
- Recently Added
- In Progress
- Closed Access
This item is closed access and not available.
Predicting natural and diverse 3D hand gestures from the upper body dynamics is a practical yet challenging task in virtual avatar creation Previous works usually overlook the asymmetric motions between two hands and generate two hands in a holistic manner leading to unnatural results In this work we introduce a novel bilateral hand disentanglement based two stage 3D hand generation method to achieve natural and diverse 3D hand prediction from body dynamics In the first stage we intend to generate natural hand gestures by two hand disentanglement branches Considering the asymmetric gestures and motions of two hands we introduce a Spatial Residual Memory SRM module to model spatial interaction between the body and each hand by residual learning To enhance the coordination of two hand motions wrt body dynamics holistically we then present a Temporal Motion Memory TMM module TMM can effectively model the temporal association between body dynamics and two hand motions The second stage is built upon the insight that 3D hand predictions should be non deterministic given the sequential body postures Thus we further diversify our 3D hand predictions based on the initial output from the stage one Concretely we propose a Prototypical Memory Sampling Strategy PSS to generate the non deterministic hand gestures by gradient based Markov Chain Monte Carlo MCMC sampling Extensive experiments demonstrate that our method outperforms the state of the art models on the B2H dataset and our newly collected TED Hands dataset The dataset and code are available at https github com XingqunQilab Diverse 3D Hand Gesture Prediction
Please use this identifier to cite or link to this item: