In the Metaverse, do we use our eyes to control everything?

Eye tracking, is the lock and key for human beings to open the Metaverse?

Have you ever imagined that you can control everything with your eyes and let objects move freely according to your own wishes?

It’s a pity that this cool, sci-fi-like scene can only appear in novels and movies in the past, like “supernatural” or “magic”. In reality, such a scenario seems impossible.

Around 2016, the emergence of the first-generation AR/VR glasses and head-mounted displays allowed people to briefly see the dawn of this scene. These devices allow people to experience immersive 3D effects. But in the first generation of products, people can still only operate through the movements of their hands and heads. And due to the technical limitations at the time, people were prone to dizziness when using it, and the sense of immersion was slightly insufficient.

But with the advent of eye-tracking technology, such an idea may finally become a reality. People can control everything with just their eyes. The position of the eye gaze is different, and devices such as VR/AR can detect it. This not only avoids the dizziness of people in the virtual reality world before, but also brings a full sense of immersion.

This technology is being favored by tech giants. Sony’s PSVR 2 and Meta’s Cambriat have all been confirmed to adopt this technology. Multiple sources show that Apple’s latest MR head display products, eye tracking function will also have a high probability of being used. use. This also indicates that eye tracking technology will become the direction indicator for the future development of MR products.

In the Metaverse, do we use our eyes to control everything?

Meta Quest 2 Source/Network

In the highly sought-after Metaverse world, MR products are regarded as the entrance to the Metaverse world widely recognized by the market. As the direction indicator of MR products, the importance of eye tracking technology to the Metaverse world is even more self-evident. metaphor.

From the above information, it can be seen that the technology giants are very fond of eye tracking technology, and have adopted the technology one after another. Eye tracking technology seems to have become the direction of the future development of MR products.

So, can eye-tracking technology, which has become the new core of tech giants’ MR products, open up a new blue ocean in the future of the Metaverse?

With questions, this article will explore eye tracking technology through the following three aspects:

1. What is eye tracking technology and how has it developed?

2. What is the value of eye tracking developed so far in various industries?

3. What market size and prospects will eye tracking technology have in the future?

1 Eye tracking, mind reading in the new era 

As we all know, the brain is the “command” of the body, and human memory, sleep, movement and language are all inseparable from the control of our brains. Of course, it also includes the control of the visual senses. So, by monitoring the movement of the eyes, it is possible to infer what is going on in the brain.

Eye tracking technology was developed based on this theory. As the name suggests, eye tracking is the process of monitoring the movement of the eyes. The core research of eye tracking is to determine where humans or animals gaze, such as real-time updates of changes in pupils and eyeballs.

Just imagine, if MR equipment can monitor the real-time dynamics of the eyes and analyze it through the system algorithm, then it can infer what the human brain wants to do at the moment, and it can also provide more and faster “ideal services” for humans.

Eye tracking technology can perform digital image processing, pupil positioning, and obtain the three-dimensional coordinates of the pupil through instruments and equipment, and calculate the gaze point of the eyes through artificial intelligence algorithms, so that the smart device knows what you are looking at. 

In the Metaverse, do we use our eyes to control everything?

Eye Tracking Rendering Source/Network

The development history of eye tracking technology can be divided into four stages: observation method; mechanical recording method; optical recording method; modern eye tracking recording method. The first three methods all appeared in the early 20th century, and now all three have been eliminated from the market due to the invention and application of various new technologies.

In the new century, more precise, more stable, non-invasive eye tracking technology was developed. The current eye tracking methods mainly include corneal reflection method, sclera-iris edge method and pupil-corneal reflection vector method.

Corneal reflection method is mainly used for eye dynamics and gaze point calibration, but the defect of large head error makes it difficult to apply to MR products that require high precision.

The sclera-iris edge method is also applied to eye dynamics and fixation point calibration. The advantage of this method is high horizontal accuracy, but the disadvantage is low vertical accuracy, large interference, and large head error.

At present, the most advantageous overall is the pupil-corneal reflection vector method, also known as the Pukinye image tracking method. This technology has the advantages of high precision, non-contact, little interference, and non-invasiveness. The core principle of this method is that the direction of sight is determined by the position of the center of the pupil relative to the corneal reflection.

In the Metaverse, do we use our eyes to control everything?

Up/Picture 1 Down/Picture 2

As shown in Figures 1 and 2, the eye forms specific bright spots through corneal reflections, the position of which can be determined by the relative position of the light source and the experimenter’s head. If the subjects kept their head still, the corneal reflex did not move when the eyeballs turned, only the pupil moved. The azimuth relationship between the pupil center and the corneal reflection changes can be clearly seen in the figure.

With the high precision and critical data this method brings, the pupil-corneal reflection vector method is also widely used in modern eye tracking devices.

And AR/VR and other Metaverse entrance products can collect high-precision key data, such as time stamp information, gaze point, eye position and pupil diameter, etc., through the eye tracking technology represented by the above methods.

Then, through the detection and systematic analysis of the above-mentioned key data, the product can know what the eyes are looking at, and then analyze what the human brain wants to do.

However, although this technology has developed into a prerequisite for “mind reading” in the new era, even at the current level of development, it is not easy for companies to truly master it. Enterprises not only need to be proficient in image vision, but also need to have in-depth research on the principles of human optic nerve and brain control, so as to further explore the value of this technology and promote practical applications.

2 Where is eye tracking used? 

As mentioned above, eye tracking technology can obtain the gaze position of the eyes by monitoring the state of the eyes in real time, and then infer the intention of the brain.

At present, people cannot communicate with their mouths and operate with their hands in their daily activities. It is foreseeable that when the eyes can replace the functions of the mouth and hands, driven by eye tracking technology, many related industries are bound to usher in a revolution.

In fact, in recent years, eye tracking technology has made many breakthroughs in all walks of life. Whether in mixed reality (MR), or in medical, automotive, retail, accessibility assistance and other fields, there are practical cases.

In the field of MR, eye tracking technology is solving the dizziness problem that MR headset products have been criticized for many years.

Previously, the multi-generation MR head-mounted display products of many major manufacturers often produced a certain dizziness after the user put it on. This is because the video screen seen by the eyes does not match the feeling of the body, and the brain is confused. The confusion of the brain is further conveyed to the whole body, causing a vertigo effect to users using MR headset products.

The vertigo has seriously degraded the experience of the device, made many consumers who were planning to buy MR products doubtful, and hindered the further development of the MR industry in the past few years.

The eye tracking technology can learn the real gaze point of the human eye, so as to obtain the position of the viewpoint on the virtual object, which is similar to the principle of the daily work of the human eye.

In the Metaverse, do we use our eyes to control everything?

Eye gaze view (left) and panorama view (right) compared to source/network

Therefore, compared with previous headsets, AR/VR products supported by eye tracking technology will greatly reduce the probability of dizziness and low immersion. In March of this year, some people in the industry tried Sony PSVR 2 with eye tracking technology in advance, and said that compared with the previous generation, it has greatly improved in terms of dizziness and smooth picture, and even “the world will be different after playing.”

Bringing people a sense of immersion and experience different from the real world is the source of the wonderful future depicted by the Metaverse. Driven by eye tracking technology, MR products are also providing a better entrance for those who yearn for the Metaverse. .

But the application of MR products is not only the full value of eye tracking technology.

In the field of marketing, through the function of eye tracking to measure visual attention, marketing subjects can more accurately interpret user behavior. Because eye tracking technology can objectively monitor people’s visual attention focus.This is crucial for the optimization of packaging design, product layout and merchandise presentation.

Pop Tarts Bites’ packaging was designed using visual merchandising techniques, resulting in an 18% increase in the brand’s total sales.

In the Metaverse, do we use our eyes to control everything?

Pop Tarts Bites Packaging

In the field of education and training, eye-tracking technology has provided new ideas with better effects for the online teaching model spawned by the epidemic.

Due to the online teaching model forced by the epidemic, US Medical Commissioner Dr. Douglas used a Miya model to combine eye-tracking technology with practical teaching methods to teach surgery in a simulated environment rather than on patients. Use a motor to assess the skill level of students.

In the Metaverse, do we use our eyes to control everything?

Eye Tracking Medical Teaching Sources/Networks

In the automotive field, eye-tracking technology is also playing an important role in identifying issues such as fatigued driving and inattentive driving. Eye tracking technology can track and judge the driver’s concentration in time, and continuously remind the driver to prevent fatigue driving.

A company called Seeing Machines has also developed over the past few years the ability to use eye tracking technology, artificial intelligence to analyze the head, face and eyes of the driver to understand the real-time situation of the driver, so as to distinguish whether they are drowsy and divided on the way. heart condition. Significantly reduce the rate of road traffic accidents. It has provided this technical service for hundreds of car companies around the world.

In the Metaverse, do we use our eyes to control everything?

Real-time monitoring of eye status source/network

In addition to being used in scientific research and business, with the assistance of eye tracking technology, people with disabilities can also use their eyes to control the computer, as an information input method to communicate with the outside world. Tobii’s eye-tracking device, TD Pilot, can help people with disabilities use their eyeballs to operate iPad apps, select objects, and even type and play through eye movements.

In the Metaverse, do we use our eyes to control everything?

Tobii TD Pilot

It can be seen that eye tracking technology has been practically applied in many industries. The industry value of this technology is being further highlighted. In the future, with the increasing demand for scenes where “eyes can be used instead of hands”, there are more places where eye tracking technology can “show its talents”.

3 Metaverse future, the blue ocean of eye tracking 

With the development of the concept of Metaverse, things that exist in the real world will be gradually digitized, virtualized, and mapped into the virtual world. This “virtual” market is gradually growing. As the entrance to the virtual world, MR devices such as AR/VR headsets will also play a pivotal role.

According to the statistics of the China Academy of Information and Communications Technology, the global VR/AR market size in 2020 is about 90 billion yuan, of which the VR market size is 62 billion yuan and the AR market size is 28 billion yuan. In the five years from 2020 to 2024, the agency predicts that the global virtual (augmented) reality industry will grow at an average annual growth rate of about 54%, of which VR will grow by about 45% and AR will grow by about 66%. reached 240 billion yuan.

In the Metaverse, do we use our eyes to control everything?

Parity Cartography

The huge MR market has given the soil for the rapid growth of eye tracking technology. According to ABI Research, a US market research organization, the revenue generated by sensors applying gesture control and eye tracking technology will be close to 5 billion US dollars.

As an indispensable part of MR equipment, eye tracking technology is expected to achieve further breakthroughs in many aspects in the future, which will undoubtedly expand the role of eye tracking technology in the Metaverse.

For example, the optics in today’s MR headsets are not yet capable of dynamic focus because the headset requires an actuated zoom display. But zoom monitors must be supported by eye tracking technology, and the system needs to know exactly where the user is looking.

However, with the advancement of eye tracking technology, future MR equipment is expected to recognize the depth change of the focus of the human eye through this technology, so that the focus will also change, and the dynamic focusing of the MR equipment will be realized.

By using the eye tracking information in the MR device, the device can also perform only “foveal rendering” to render those elements in the observed environment. This not only avoids the GPU overload caused by too many resources required for rendering, but also accurately brings a higher quality visual experience, creating a more immersive Metaverse environment.

In addition, the tracking of vision by eye tracking will also help various devices such as special MR to make more accurate judgments of user intentions in the future. Devices and systems will be able to plan the user’s next behavior based on the user’s gaze point, and create more appropriate settings for the Metaverse world.

Eye tracking also has good application prospects for improving interaction efficiency. With eye tracking speeding up the operation speed and accuracy, users can also actively use their gaze points to complete the real world and virtual world more easily. Various operational tasks in the world.

Whether it is remote surgery, remote driving, etc. in the real world, or various operations in the virtual Metaverse, the development of eye tracking technology will bring more convenience to human beings.

Although the current market size of eye tracking cannot be compared with the overall market of MR, as a “precision scalpel” in MR and even Metaverse equipment, it is destined to gain broad market share in the future with the development of MR and the evolution of Metaverse. Development world.

As Palmer Luckey, founder of Oculus, the well-known Meta-owned MR equipment company, said: “Eye tracking is the heart of the virtual world, and future visual headsets will definitely support eye tracking technology.”

As the core of the entrance to the future Metaverse world, eye-tracking technology is destined to be promising in the vast market scale in the future.

4 write at the end 

With the continuous iteration and development of the virtual reality industry, and the rise of the concept of the Metaverse, the statement that MR equipment is considered to be the next generation computing platform is gaining more and more recognition.

MR headsets are also developing towards being lighter, thinner and more convenient. According to relevant statistics, MR headsets are expected to sell more than 100 million units in 2025. In the future, the scale of the MR industry is expected to surpass that of smartphones and become a portable computing terminal for every person.

In the Metaverse, do we use our eyes to control everything?

Future MR glasses source network

It is foreseeable that in the near future, more and more things will be done by humans through eye tracking technology on MR devices. For example, creation and design can manipulate the input terminal through the eyes; doctors can also use related smart devices to complete a refined remote surgery with eye movement control. Interacting with virtual worlds will also become as easy as making a phone call now.

Controlling everything with your eyes will no longer be just a dream, but a part of your life. The future has come and is coming. Eye tracking technology is liberating human hands. Will the era of “eye control” instead of “hand control” be far behind?


1. “Basis and Application of Eye Movement Analysis Technology” Yan Guoli;

2. “The Method of Eye Tracking Technology” Chen Rui;

3. “Where is the charm of eye tracking” 93913 virtual reality;

4. Tobii Pro, “The first understanding of eye movements, the background knowledge of human eye movements you need to know”;

5. “The new generation of XR head display comes standard with a detailed explanation of the value and challenges of eye tracking technology” VR gyro.

Posted by:CoinYuppie,Reprinted with attribution to:
Coinyuppie is an open information publishing platform, all information provided is not related to the views and positions of coinyuppie, and does not constitute any investment and financial advice. Users are expected to carefully screen and prevent risks.

Like (0)
Donate Buy me a coffee Buy me a coffee
Previous 2022-04-20 10:12
Next 2022-04-20 23:35

Related articles