What we learned about 3D avatars in 7 years of making them
Christoph Niemann is an illustrator whose work was featured on over thirty covers of the New Yorker magazine. In the first episode of Netflix's docuseries "Abstract,” Niemann introduced a concept called abstract-o-meter. It suggests that every idea has a just-right way to represent it. It can’t be too abstract nor too realistic. For example, no one would know that you meant to talk about love if you used a red pixel to represent it. The same thing applies to a realistic drawing of heart, which looks more disgusting rather than lovely. Hence when we talk about love, we use the heart-shaped symbol that everyone is familiar with.
The same concept applies to 3D avatars that we use to represent our identities in virtual worlds and to show our emotions to others.
Abstract 3D avatars are too simple
It's hard to show our expressions in games with a simple and abstract visual style. In Minecraft, the avatar's face measures 8x8 pixels. It’s barely enough room to fit eyes and a mouth – not enough to represent a wide range of emotions. In fact, Minecraft doesn't offer any expressions besides blinking with compatible character skins.
Showing your identity in a world made of big pixels isn't a simple task either. You can give someone a general idea of how you look and what colors you like to wear. But it would be hard to distinguish, for example, a black denim jacket from an elegant blazer.
Hyper-realistic 3D avatars are too realistic
On the other side of the spectrum, we have hyper-realistic avatars. In Rogue One, Disney created a digital version of Wilhuff Tarkin based on the actor who played him a few decades before – Peter Cushing. This was possible thanks to advanced motion tracking technology and an actor (Guy Henry) who learned how to behave and talk like Cushing.
The same technique is used in modern AAA games, like Cyberpunk 2077. The challenging part is to make the representation perfect. Otherwise, we experience an effect called the uncanny valley. If the humanoid character or object doesn't perfectly resemble a human, we tend to have negative emotions. In other words – we are creeped out.
In the movie and gaming industries, studios have access to and can afford advanced tracking solutions. But they also spend thousands of hours on post-processing to make the characters look like real humans. For consumers, there’s no way to use a hyper-realistic avatar based on the user’s appearance. Modern games often have advanced character creators, but customization options are limited just to pre-set values and assets.
These avatars are also not able to represent the player’s expressions. And there is no affordable face tracking devices available on the mass market that could produce results comparable to the recent Star Wars movies. Not to mention the challenge of rendering super-detailed 3D characters, especially in games like Call of Duty: Warzone with a hundred players on one server.
The just-right solution for 3D avatars
The just-right solution for avatars is a simple one. Going back to the uncanny valley concept – it's called a valley because we dislike only humanoids that look almost like humans. But we tend to like slightly abstract representations of humans, like cartoon and game characters. These avatars are more detailed than the Minecraft ones, making it easier to show one's identity and emotions. On the other hand, they are not too detailed, making it simpler to show basic emotions without creeping someone out. They are also less challenging to render for average hardware, even in large-scale games like Fortnite.
The current state of consumer technology allows us to use these simpler avatars to have social interactions in the virtual worlds. For example, there are software-based solutions for emulating mouth movement, like Oculus Lipsync. We can also get a good representation of body movement with the controllers that come with virtual reality headsets available today. This is essential for getting non-verbal communication in virtual worlds right.
Finding the right balance for 3D avatars
But there's one more reason why less realistic avatars are just right, and we learned it through our company’s long history of making avatars. When we started Ready Player Me (back then, Wolfprint 3D) seven years ago, we built full-body scanners for creating hyper-realistic human models desired by movie and gaming studios. When Facebook acquired Oculus in 2014, we realized that 3D avatars will play an important role in the future. We shifted our focus and decided to make it easy for anyone to create their own 3D avatar.
Our first step in that direction was easy-to-use, egg-shaped face scanners. We shipped them to many places worldwide, including museums in Estonia and Finland, E3 game shows in Las Vegas, and Paramount Studios in Hollywood.
The next step was to make scanning even more accessible. We decided to use the data from tens of thousands of high-resolution scans to build a software-based solution. With just fifteen photos and a phone app, anyone could create a realistic 3D model of their face.
At this stage, we learned that people didn't want an avatar that looks exactly like them – they wanted something more. It's like with Instagram profiles — no one posts photos of their dirty dishes (please, prove me wrong). Instead, we choose to create an idealistic version of our lives featuring pictures from trips to Bali, plates full of sushi, and selfies from Coachella.
Where are we now and what's the future of 3D avatars
Learning this, we set Ready Player Me on its current tracks and made a simpler avatar creator based on a single selfie. It allows anyone with a camera and access to the Internet to create an idealized, 3D version of themselves. All with the just-right balance – not too abstract, not too realistic. Cristian Anton, Co-founder and CEO of MeetinVR, said in an interview with Ready Player Me's CEO that "what we have here [...] is like the perfect balance, at least for what is possible with the current technology."
Ready Player Me avatars are our current just-right solution for avatars. They are available in 3,000+ apps and games, allowing people to have social interactions in virtual worlds while keeping a consistent digital identity. Think of it like this: in the morning, you can hop into a meeting via MeetinVR; in the afternoon, you join a meetup on Mozilla Hubs; in the evening, you hang out with your friends in VRChat or stream your Beat Saber session via LIV. All using the same avatar.
If future technology allows for it, we might see the shift towards more realistic avatars. But right now, we are in a perfect spot for representing our identities and showing emotions without creeping anyone out.
Are you a developer?
Do you want to join us in the mission of building the metaverse? Become a Ready Player Me partner to integrate our avatars with your app or game.