What will Apple’s highly anticipated mixed reality glasses be capable of? As I’ve written before, Apple is likely hiding clues about the design and functionality of the glasses in plain sight. It’s putting features in existing products that will only reveal their true usefulness in a mixed reality (XR) context.

Memoji–the animated emoji you can personalize to reflect your look, moves, and emotions–may be an example of this. You can already speak through your Memoji during messaging sessions or FaceTime calls on your iPhone. In mixed reality, your Memoji will likely become your 3D avatar that represents you in immersive work or social meet-ups.

Avatars will likely play a big role in mixed reality, or “spatial computing” as it’s sometimes called, at least in the beginning. Consumers will expect to be able to have virtual meetings with family members that appear to be sitting around the kitchen table in front of them. Business users will want to see their workmates gathered around them in virtual space. It’s very likely that Apple will extend its FaceTime video conferencing into mixed reality to accommodate such get-togethers.

In an ideal world, we might see the participants in such meetings as live action holograms gathered in front of us. But there remain some difficult hardware engineering barriers to enabling that in any kind of polished way on mobile, wearable devices. Until that becomes possible, we’re going to be relying on avatars to represent us in virtual meetings.

This might not be a bad thing. As Computerworld columnist Mike Elgin points out in a recent column, some people might actually prefer showing up as an animated avatar as opposed to showing their real face to the glare of the Zoom camera. Avatars, Elgin suggests, might put a little distance between the participant and the public glare of Zoom calls, while still allowing for the communication or collaboration that needs to get done. (Actually, Elgin wrote about the potential use of Memoji/FaceTime in XR back in 2020.)

But not all avatars are created equal. Most of the virtual meeting avatars we’ve seen so far seem janky or wooden or flat (see Meta’s Horizon on Meta Quest 2). They lack the capacity to believably express body language and facial expression.

Apple may have a built-in advantage with its Memoji feature, which lets you create an animated emoji that looks and moves like you.

Your Memoji moves like you because the 3D depth camera on the front of your iPhone sends out little pulses of light that detect the real-time movements of numerous surface points on your face. It then uses that data to recreate your head movements and expressions in your animated likeness, which you can use as your proxy in either recorded or real-time communication in either Messages or in FaceTime.

With Apple’s XR glasses, a different set of sensors will likely detect your expressions and head movements, but the same data (and probably a lot more) could be collected and used to create your avatar for meetings in a sort of FaceTime XR.

Lest you think this all sounds futuristic, it may be coming faster than you think. A number of well-sourced and trustworthy reports (along with a number of not-so-trustworthy reports) have leaked information about Apple’s work on its XR product.

Digitimes Asia, which has a mixed record on Apple rumors, cited unnamed supply chain sources in a Monday story claiming that Apple has already conducted the “second-phase engineering validation and testing” of its AR headset. If true, this suggests an already-baked tech product that is making its way toward market-readiness. The Digitimes sources say the headset will show up later this year.

In the Apple rumorsphere, a consensus of opinion seems to point to two separate Apple XR products–one sooner, one a little later. The first (in 2022 or 2023) is a headset that can run either AR apps or VR apps. Then, Apple will release a more refined pair of AR or mixed reality glasses that can be worn for extended periods.

The volume of the chatter is increasing lately—and for good reason. Many people, including me, believe we could be seeing the start of a whole new personal computing paradigm–the thing that necessarily has to come after the smartphone.

The vision is that one day you’ll be able to put on a svelte pair of mixed reality glasses (that look very much like the reading glasses you’d wear anyway, or even contact lenses) and discover that your computer’s operating system has spilled off the screen of your smartphone and now magically inhabits much of the visible world in front of you. And it does so in a dynamic way; that is, it’s smart enough to hide the stuff you don’t need and only surface the digital resources (data, images, video, etc.) exactly when you need them. It might also place those digital things in the exact right spot in your field of view. A name reminder might appear above the head of a business contact approaching, or a mapping app might flash an arrow just above the left turn you need to make. Of course, the best use cases haven’t been dreamed up yet, and we probably won’t see them until we start living with XR glasses.

But there’s a lot of engineering work to be done. Consumers will likely only embrace true spatial computing when the hardware devices (headsets, goggles, or glasses) become small enough to stay out of the way. On the software side, users won’t tolerate much kludginess or bloat in the experience–the technology literally proposes to assert itself over our sight and how we experience and make sense of the world.

It could be another decade before tech companies get these things right. As for Apple, it became the most valuable company in the world mainly by anticipating then delivering devices that people love—then depend on. Whether it can repeat that magic in the realm of mixed reality remains to be seen. But its Memoji could give it a big advantage.


Fast Company Senior Writer Mark Sullivan covers emerging technology, politics, artificial intelligence, large tech companies, and misinformation. An award-winning San Francisco-based journalist, Sullivan's work has appeared in Wired, Al Jazeera, CNN, ABC News, CNET, and many others. More

More Top Stories: