VEPHand: View-Efficient Photometric Hand Performance Capture at Scale
Abstract
High-fidelity 3D hand capture is fundamental to digital human creation, but achieving robust results from practical multi-view systems—which balance rich photometry with inherent geometric underspecification—remains challenging. This paper introduces an end-to-end pipeline for dynamic hand performance capture and registration designed for such conditions, utilizing a 20-camera photometric setup. We address core difficulties through two primary innovations: first, a mask-free neural reconstruction method that leverages scene parameterization and scenario-specific density regularization to robustly extract detailed hand geometry and appearance from unmasked images despite limited view overlap and potential background clutter. Second, we propose a volumetric physics-inspired registration framework that aligns these reconstructions to a personalized hand model built on a consistent tetrahedral mesh, enabling the capture of fine intrinsic deformations and ensuring plausible results under severe articulation and self-contact. We demonstrate the scalability and robustness of our automated pipeline on an extensive dataset of over 8,000 sequences, showcasing its effectiveness for single hands, intricate two-hand interactions, and detailed hand-object manipulations, significantly advancing the state-of-the-art in practical, high-quality hand capture.
Research Areas
