TaoAvatar Lifelike Full-Body Talking Avatars for Vision Pro

In the past few months, we have covered plenty of apps that let you interact with AI avatars. TaoAvatar can make them a whole lot more photorealistic. It generates 3D full-body avatars with controllable pose, gesture, and expression. The researchers created a 3D digital human agent on the Apple Vision Pro. It interact with users through automatic speech recognition with LLM and TTS.

As you see in the above GIF, facial expressions and gestures are dynamically controlled by an Audio2BS model. TaoAvatar  has a frame rate of 90fps. For its dataset, it uses eight multi-view image sequences captured with RGB cameras in 20 fps, and a resolution of 3000×4000p.