Why eye tracking will be standard on all VR headsets

Since the start of the digital revolution, new technologies have continually replaced physical objects with software services, progressively virtualizing how people communicate. Airmail has transformed into instant messaging, newspapers into websites, and face-to-face meetings are being replaced by online hangouts, telepresence, and VR.

The more we virtualize, the easier it becomes for people to collaborate remotely across time zones and distance. In this post, I take a look at the impact of virtualization on remote collaboration, which has become more intuitive and accessible over time, and why application developers are building for eye tracking.

A few years ago, I was helping my parents clear out their attic. Amid the memories and dust bunnies, I found a stack of cassette tapes dating back to the 1970s. What I had uncovered was the correspondence between my parents and my grandmother. At the time, they lived on separate continents and communicated with what they refer to as tape mail. It was their unique means of keeping in touch in a way they felt was more profound, more meaningful than written letters.

During this era of Woodstock and the first oil crisis, the cost of a New-York-to-London round-trip would be today’s equivalent of about four thousand US dollars. Handwritten or typed memos were the primary form of business communication. My parents’ innovation made voice communication affordable at a time when the sound quality of long-distance telephony was poor. Calls needed to be planned, they were costly, and conversations limited to gathering the latest news.

The past few months have reminded us all of the human need for personal connections — a need reflected by the dramatic increase in remote VR collaboration. In March 2020, VR Chat usage was up by 48% compared with the same period in 2019. Steam noted similar increases with AltspaceVR up by 13%. For me, these figures illustrate the potential of VR and how it can bring people together in a way that strengthens relationships.

Today, in what seems to be an ongoing family tradition, my kids and their grandparents also live on two separate continents. Usually, we get together once a week on a video call, during social distancing times, it has become a daily event. The availability of limitless voice and video, multiple devices, and robust network connections allow us to have fun together — to communicate naturally.

And while I am grateful to have a job and the technology that supports business as almost usual, I cannot help but wish that next-gen VR headsets and applications were already mainstream. It would help my kids to learn in an interactive way that ensures they retain information. And for me, I could hold meetings and collaborate remotely with the additional creativity and freedom promised by widespread VR.

The virtual war room

Right now, I’m about to start up a new project. A few years ago, we would have customized a war room at the office, a place for the team to get together, to be creative, discuss, test, and refine. Today, just like many other modern enterprises, the experts in my team are spread across time zones. Collaboration requires patience and creativity. VR frees us from the conventions and limitations of physical spaces, distance, and time. In our VR war room, we can bring all the tools we need, presentations, videos, live content, the most recent 3D model of our product, the event booth for CES next year, and our new design ideas.

Like so many other virtual solutions, VR brings the benefit of convenience, reduced travel, and sustainable collaboration. But VR experiences exceed others in their flexibility and hyper-realism — because it enables us to do things that are impossible in the real world.

Enterprise-grade apps with convincing avatars

As I continue to daydream about working in such a way, the truth is, remote VR collaboration is gaining a foothold in the entertainment world. And owing to the technologies that have enabled avatars to evolve from gimmicky cartoons into realistic digital twins, VR applications are beginning to creep into the C-suite. Continued adoption depends on useful enterprise-grade apps and convincing avatars. Eye tracking supports both, and this is why I am convinced it will be standard on all headsets. It removes the uncanny stare, giving avatars natural eye movements and side glances eye ensuring avatars continue to develop along the path of deepening realism. And by removing the need for unnatural head movements to, for example, select objects, eye tracking can improve the level of intuitiveness in VR applications.

HTC recently announced the open beta of VIVE Sync, a VR collaboration and meeting tool built specifically for enterprise communication. Currently free for businesses, it supports Tobii eye tracking to enable avatars with natural interaction.

Needless to say, I couldn’t wait to try it out. I wanted to see for myself how Tobii’s eye tracking technology supports intuitiveness in remote collaboration and brings life to a new generation of VR avatars.

So, when my colleague Maggie wanted to talk about how eye tracking works in a VR headset, instead of having a video meeting, as we usually do, we got together with Vive Sync. I had an eye tracking equipped headset, but Maggie didn’t. The recording of our meeting clearly shows the difference in our eye movements — mine are tracked in real time, whereas Maggie’s are simulated.

Similar to the way haptic controllers add touch, eye tracking adds sight to the VR experience, capturing eye contact, how we blink, perhaps the roll of an eye — all the content-carrying gestures that are critical to meaningful communication.

If you’re interested in my high-level description of how eye tracking works in a VR headset, you can watch our 3-minute meeting:

Starting to build for eye tracking

Some VR devices already include eye tracking, and I am certain that the technology will be standard on all new designs, not just for the apps they enable but because the technology is a prerequisite for well-optimized headsets with foveated rendering (check out Realistic virtual vision with Dynamic Foveated Rendering if you want to know more about this technology).

To accelerate the creation of new applications, Tobii’s Developer Zone provides SDKs and demos that help developers create avatars that emulate a user’s eye movements or map their gaze point to an object on the screen. In the meantime, I cannot wait to try out VR with my kids and their grandparents on one of our calls.