RMAvatar: Photorealistic Human Avatar Reconstruction from Monocular
Video Based on Rectified Mesh-embedded Gaussians

method

framework of our rmavatar.

Abstract

We introduce RMAvatar, a novel human avatar representation with Gaussian splatting embedded on mesh to learn clothed avatar from a monocular video. We utilize the explicit mesh geometry to represent motion and shape of a virtual human and implicit appearance rendering with Gaussian Splatting. Our method consists of two main modules: Gaussian initialization module and Gaussian rectification module. We embed Gaussians into triangular faces and control their motion through the mesh, which ensures low-frequency motion and surface deformation of the avatar. Due to the limitations of LBS formula, the human skeleton can only control rigid transformations. We design a pose-related Gaussian rectification module to learn non-rigid deformations of cloth and hair, further improving the realism and expressiveness of the avatar. We conduct extensive experiments on public datasets, RMAvatar shows state-of-the-art performance on both rendering quality and quantitative evaluations.

Comparison Experiments

Novel view synthesis on PeopleSnapshot: Our method is able to reconstruct intricate texture details.

Novel view synthesis on ZJU-MoCap: Our method reconstructs complicated cloth textures.

Avatar animation on out-of-distribution poses: Our method generates consistent representations for avatars on challenging poses.

Related Links

GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians.

SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting.