| | |
| | |
Stat |
Members: 3667 Articles: 2'599'751 Articles rated: 2609
08 February 2025 |
|
| | | |
|
Article overview
| |
|
Towards a Pipeline for Real-Time Visualization of Faces for VR-based Telepresence and Live Broadcasting Utilizing Neural Rendering | Philipp Ladwig
; Rene Ebertowski
; Alexander Pech
; Ralf Dörner
; Christian Geiger
; | Date: |
4 Jan 2023 | Abstract: | While head-mounted displays (HMDs) for Virtual Reality (VR) have become
widely available in the consumer market, they pose a considerable obstacle for
a realistic face-to-face conversation in VR since HMDs hide a significant
portion of the participants faces. Even with image streams from cameras
directly attached to an HMD, stitching together a convincing image of an entire
face remains a challenging task because of extreme capture angles and strong
lens distortions due to a wide field of view. Compared to the long line of
research in VR, reconstruction of faces hidden beneath an HMD is a very recent
topic of research. While the current state-of-the-art solutions demonstrate
photo-realistic 3D reconstruction results, they require high-cost laboratory
equipment and large computational costs. We present an approach that focuses on
low-cost hardware and can be used on a commodity gaming computer with a single
GPU. We leverage the benefits of an end-to-end pipeline by means of Generative
Adversarial Networks (GAN). Our GAN produces a frontal-facing 2.5D point cloud
based on a training dataset captured with an RGBD camera. In our approach, the
training process is offline, while the reconstruction runs in real-time. Our
results show adequate reconstruction quality within the ’learned’ expressions.
Expressions not learned by the network produce artifacts and can trigger the
Uncanny Valley effect. | Source: | arXiv, 2301.01490 | Services: | Forum | Review | PDF | Favorites |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
|
| |
|
|
|