One of the standout features of the Apple Vision Pro is the realistic avatars that represent you on a FaceTime call. They allow you to appear on a phone or laptop as yourself, despite calling while wearing a visor. Meta is now close to launching its own version for the Quest.
“Codec Avatars” have been in development by Meta’s research team for some time, designed to provide a photorealistic version of the user while they are in a fully immersive environment.
A recent investigation of the Quest firmware by a user called Luna revealed the potential for a rollout of the new realistic avatar system — replacing the cartoonish avatars in use today.
In March 2023 CTO Andrew Bosworth warned during an Instagram AMA that despite making progress they were "still years away" from being able to launch them. While the appearance in the firmware doesn’t mean the launch is imminent, it does suggest a more rushed timeline.
What are ‘Codec Avatars’?
Meta first announced plans for more realistic avatars in 2019 with the launch of a research and development project aimed at improving remote communication.
Over the past four years, the company has gradually revealed fidelity updates and new features, but so far user representation in the virtual world has been largely cartoony or game-like.
Version 62 of the Meta Quest firmware, only available in the public test channel and not as a default option, points to new features—including a line mentioning codec avatars for the first time.
It is likely the initial release will work on head and shoulder shots, similar to what might appear on a FaceTime or Zoom call. This is similar to the way Apple's Vision Pro personas appear.
Unlike Personas, which use front display and depth sensors, the Codec Avatars are fully rounded virtual characters depicted as fully photorealistic. They are encoded virtual 3D models using data captured by a smartphone and processed using AI models in the cloud.
This would allow for them to fully integrate into a virtual world, rather than simply appear on a virtual screen on the headset or on a third-party device. An example of the technology has already been shown publicly at Meta Connect.
For future Quest headsets
Bosworth said in a video in March: "The challenge is how we get the capture from a cellphone in a short period of time i mixed lighting. How do we get the sensors to pick up enough data to animate the avatar. Lots of engineering challenges as well as research challenges."
This suggests Codec Avatars are likely going to require better hardware than is available on current Quest headsets and may be reserved for a future Quest Pro 2.
