Voice AI and the Metaverse Shopping Experience

What will the metaverse shopping experience look like in another year, or two, or five? And how will conversational AI play into the ecosystem? We consulted an expert to find out.

Kane Simms is CEO of VUX World, a strategic AI consultancy that helps companies boost revenue, cut costs, and improve customer experience. We asked Kane how he expects retail in the metaverse to develop—and what role voice AI will play in the process. Here’s what we learned.

Whatever else it is, the metaverse is undoubtedly a commercial enterprise. This immersive virtual space—a kind of 3D internet, accessed through virtual reality, augmented reality, and other devices—leapt into mainstream public consciousness in October 2021. That’s when Mark Zuckerberg announced Facebook’s rebranding as Meta, asking us to think of it less as a social media company and more as a metaverse pioneer.

In an earnings call a few months earlier, Zuckerberg hinted at the business model underlying this shift. Advertising would remain “an important part” of the company’s profits in the metaverse. So would digital retail—leading to widespread speculation about what the metaverse shopping experience will look like.

It’s a vital consideration for retailers. Analysts theorize that people will spend increasingly more time in these persistent virtual spaces. Gartner predicts that 25% of consumers will visit the metaverse for at least an hour a day by 2026, whether that’s for work, socializing, entertainment, education, or, of course, shopping.

But to answer the question, we need to know how the metaverse will develop. For all the hype surrounding this topic, the “place” itself is still mostly hypothetical; the metaverse does not yet exist in a stable, universally accepted form. Retailers can start imagining the metaverse by looking at the development of similar technology trends in the past, from the internet in general to the rise of social media.

Charles Cadbury, CEO of conversational AI agency Say It Now, has observed that most new interactive digital technologies follow a similar progression:

1. Novelty abounds. The new technology is cool, and early adopters focus on that coolness factor. Gaming and socialization are the main activities. The metaverse is still in this stage as we publish.
2. Utility emerges. Play reveals more practical possibilities. In the metaverse, maybe that means far-flung colleagues meeting in a virtual space to collaborate. Maybe it means immersive training and educational content. It will almost certainly involve virtual reality stores that render products—both real and digital—in 3D.
3. Monetization arrives. Where there’s utility, there’s value, which is to say money. Brands show up where people cast their attention. Products and services follow. The internet is clearly at this stage; so is social media, and so are mobile apps. The metaverse will inevitably end up as a space for commerce, too.

Of course, these stages of digital maturation don’t occur along a straight line. They blend and merge. Given the rise of in-app purchases, gaming is commerce, which you could argue has utility (at least for the game studios). Still, this three-stage model provides a hazy picture of development for the internet, social media, and mobile apps. It will probably apply to the metaverse, too.

But to return to our central question: What will the mature metaverse shopping experience look like? Here are some of Simms’ ideas on the topic.

Defining the Metaverse Shopping Experience

First, a clear definition: Metaverse shopping is any commercial transaction made in immersive, online, virtual worlds. That could include:

• Digital recreations of physical retail stores. Say you want a pair of Nike shoes. Right now, you go to the website (or, more likely, Zappos or Amazon). You make your decision based on two-dimensional pictures and customer reviews. In the metaverse, you might enter a 3D Nike store, talk to sales associates, pick up digital models of shoes, and even have your avatar try them on. Maybe you walk your items up to a virtual counter to make the purchase; a few days later, the shoes arrive on your doorstep, just like with traditional e-commerce.
• Virtual brand experiences. In the metaverse, retail stores can be hubs for direct experiences unavailable in the real world. Rather than just looking at Patagonia jackets, for instance, you might be able to virtually climb a section of Kilimanjaro—while noting what that jacket looks like on your avatar during the trip.
• Streamlined app purchases. Say you’re logged into your virtual office, VR headset on, coworkers from all over the world sharing the same space. Maybe you’re working with Google Sheets or Excel and you need mail-merge functionality. In the metaverse, you could access a mail-merge add-on app by calling up an app store in mid air. You buy the add-on, charge it to the company account linked to the system, and continue working—without leaving the virtual space you already inhabit.
• Buying virtual goods. Globally, in-app purchases were already worth more than $100 billion in 2021—and that figure is expected to rise at a compound annual growth rate of over 20% through 2027. These transactions include the purchase of digital goods, like new outfits for gaming avatars. In the metaverse, the options for persistent virtual clothes, properties, and items will have no limit—and these purchases will be a powerful revenue source for retail brands. As Zuckerberg said in his July 2021 earnings call:

“I think digital goods and creators are just going to be huge…in terms of people expressing themselves through their avatars, through digital clothing, through digital goods, the apps that they have, that they bring with them from place to place.” — Mark Zuckerberg on the Metaverse

These speculative examples only scratch the surface, of course. If there’s one thing the growth of the internet has taught us, it’s that innovation will surprise you. But no matter how metaverse shopping develops, we’re confident of one element of the technology: conversational AI, specifically voice AI, will be a major contributing factor to the metaverse shopping experience—as it already is in today’s retail environment.

How Voice AI Will Support Retail in the Metaverse

Voice has a good chance of becoming the predominant interaction modality in the metaverse. As we publish, the metaverse doesn’t render arms or legs on your avatar. It can’t, because you’re not wearing sensors on your elbows, ankles, fingers, etc. So the software has no way to tell how you’re moving, and it can’t mirror your actions in the virtual world.

Think about the ways we interact with the digital world: keyboards, mouses, taps, and swipes. None of those are currently available in the metaverse. Maybe a computer vision sensor will eventually allow the subtle movements of your fingertips to register in a metaverse environment. Maybe you’ll put on sensing gloves, or a whole bodysuit. But for now, voice is the simplest way to interact with metaverse systems (see sidebar).

The Key Question About Metaverse Shopping

How will you pay for items in the metaverse? On a typical e-commerce website, you enter credit card details or pay with PayPal or a similar service. You also have to navigate security, whether that’s biometrics (like your thumbprint) or two-factor authentication. In the metaverse, you don’t have a keyboard or a touchscreen, so the question remains: How can you authorize payment while keeping that transaction secure? A likely solution is voice biometrics, in which the unique sound qualities of your voice act as authentication of your identity. This technology has already worked well in voice banking and contact centers. Voice biometrics trials on Amazon Alexa and Google Assistant have been promising. For retailers, voice biometrics aren’t just helpful as an authentication device. The voice can also identify individual consumers, enabling personalized service. For instance, if a customer has looked at a certain handful of products online, retailers can display similar products when that customer enters their metaverse store—identifying them by the sound of their voice.

If voice is your primary input tool, it will also become the system’s main output (along with visuals, of course). And that will require voice AI, artificial intelligence systems that understand natural language and reply to users through speech. Here are some scenarios or use cases for voice AI in metaverse shopping—and the branding that leads consumers to choose your store over a competitor’s:

1. Branded Virtual Characters

There’s no need to hire a celebrity spokesperson in the metaverse. Instead, brands will build original virtual characters powered by conversational AI. These characters are already available. Ilya Gelfenbeyn, who headed Google’s conversational AI product, Dialogflow, started a virtual character company called Inworld in 2021, for example.

The rise of branded voice assistants has prepared us for these virtual characters, which are best understood as an evolution of assets like Amazon Alexa, Google Assistant, Bank of America’s Erica, or Capital One’s Eno.

The value of these branded voice assistants is that they create a consistent brand experience across customer touchpoints. Whether you engage with Bank of America through its website, app, or contact center, you’ll speak to the same voice assistant—one that offers the same level of service and the same, reliable experience. That’s key to proper branding.

In the metaverse, these brand characters will have a 3D physical presence as well as a voice. But the voice is the center of that character’s functionality, and voice AI is the technology that will allow them to provide satisfying interactions.

2. Automated Sales Interactions

As we publish, retailers are struggling to hire and keep employees—so much so that 83% of retail companies said their main investments were in employee recruitment and retention in a 2022 Deloitte survey. Now, imagine staffing stores for a public that spans the globe. That’s the challenge of metaverse retail.

It won’t be possible to place human sales agents in metaverse shopping outlets. To meet the scale the metaverse makes possible, companies will need voicebots, which go beyond one-on-one retail transactions for limitless scalability.

3. One-to-Many Customer Service

The metaverse gives retailers access to a global market, and to keep up, you’ll need to scale customer service alongside sales and marketing. Voicebots are already performing this service for websites, mobile apps, and contact centers. They’ll do the same in the metaverse.

But these voicebots have their limits. It’s simple enough to answer frequently asked questions like “What’s your delivery time?” and “How can I make a return?” But more complex questions (“Where do the materials in this product come from?”) are more challenging for today’s conversational AI. That’s not to say they can’t be answered, just that some brands lack the data or content to answer these questions today—questions that you only realize people need answering when you allow them to have conversations with you. And so metaverse customer service AI isn’t likely to match human performance any time soon.

The Challenges of Implementing Voice AI in Metaverse Shopping

To understand how voice assistants will work in the metaverse, look at how they operate in apps and automated contact centers today. These use cases involve two challenges: First, the voicebot has to understand what customers say. Second, it has to provide a satisfying answer. Customer service in the metaverse will face the same challenges—as well as new ones.

When you present a virtual being, the human expectation will be that they can interact as humans. A customer will naturally wish to say anything to your virtual customer service agent. They’ll expect a human-like response.

As advanced as natural language generation (NLG) AI has become, it still comes up with gibberish sometimes. Artificial intelligence simply isn’t ready to carry on a human-like conversation on every subject. In a contact center, or on an app, conversational AI gets around this problem by providing prompts, accessing pre-written responses, or constraining conversational pathways through a visual interface.

These techniques will be less effective in the metaverse, where customers will engage with a lifelike being, and will expect lifelike interactions. And along with the language generation problem, there’s the issue of the agent’s behavior: Where do the eyes look? What expressions are on its face? What does it do with its hands?

That’s an additional visual design challenge, stacked on top of the challenges we’re currently confronting in conversational design. In short, virtual characters for metaverse shopping aren’t quite ready for prime time—but, if the rapid advances in conversational AI are any indication, they will be soon. If they’re to build stronger customer relationships, however, virtual characters will need to be recognizable, unique, and consistent.

Branding Consistency in Metaverse Voice AI

A consistent brand experience will be more crucial in the metaverse than it is in traditional customer touchpoints. In today’s customer service apps and automated call centers, customers are focused on utility over experience; as long as their issue is resolved, they may not care too much about the virtual agent’s voice.

The metaverse flips that ratio. The defining characteristic of the metaverse is its immersion, and audio is an ever-present element of the human experience. In fact, we expect sound to play a bigger role in the metaverse than it has in any form of previous media.

That makes your brand voice—literally, as expressed by metaverse user interfaces and virtual characters—more important than ever. If you create a branded virtual being that has conversations with millions of people all over the world every day, you need a curated voice that expresses your brand perfectly.

That’s where ReadSpeaker can help. We help you build brand affinity with a unique, lifelike synthetic voice tailored to your brand personality. With a branded text-to-speech (TTS) voice, you create a consistent experience across touchpoints—in the metaverse and beyond. This voice becomes a core asset of your brand, a sort of audio logo, and as central to your identity as your color palette.

As you start to think about the metaverse shopping experience that will take your brand into the future, keep custom branded voices from ReadSpeaker in mind.