GHARP: Apple’s real-time 3D head animation from few images
In brief
- GHARP animates 3D heads in real time from a few images and an expression signal.
- The system separates costly reconstruction from lightweight animation for optimal performance and quality.
- Results: record quality on Ava-256, 13× faster and 8× fewer Gaussians than an A100 GPU.
Apple unveils GHARP, a method for real-time 3D head animation using a few images of a subject and an expression signal. The approach splits the process into an offline identity stage—building a detailed geometry and appearance model—and a lightweight runtime animation stage optimized for mobile devices.
GHARP leverages a semantically structured latent space from a pretrained reconstruction model, confining expression changes to the face area. This enables merging multiple input views into a compact canonical Gaussian representation. Yet, the method inherits a limitation from expression-driven avatars: body pose and clothing are not specified, leading to ambiguities. To fix this, researchers add a body alignment network that corrects inconsistencies between the target and reference images.
Tests show state-of-the-art quality on the Ava-256 benchmark, with up to 13× faster processing on an A100 GPU and 8× fewer Gaussians. This breakthrough could speed up the development of interactive 3D avatars for mobile or immersive applications.
What it changes for you
- Media & EntertainmentContent creators could integrate more realistic and fluid 3D avatars into their productions, cutting costs and rendering time.
- Video GamesGame developers could offer more dynamic and customizable playable characters or NPCs, with smoother facial and body animations.
This summary gives you the essentials. The full article, its figures and quotes are on Apple Machine Learning Research.
Read the original articleAI-generated summary of the source, checked by rules. How we write · Report an error