Virtual Humans: From the Game to the Bigger World

Have you ever wondered what the future digital life will be like?

The movie “Ready Player One” shows one of these scenarios, in which people explore various possibilities in a world where virtual and real are combined through “avatars”.

In the movie, although the heroine Artemis is not perfect in reality, she is a heroic heroine in “Oasis”, which is inseparable from an important technology – virtual human.

Today, Wei Xi will talk with you about virtual human technology and the future it represents.

Virtual and real on the same screen: starting from a special party

During the May 1st holiday, CCTV’s May 4th Gala this year was particularly different.

An important highlight of it is that three avatars and many young actors boarded the stage of the performance. The three avatars are Tong Heguang, Xingtong and Ji Li of Tencent Interactive Entertainment.

Virtual Humans: From the Game to the Bigger World

Accompanied by the familiar melody of “New Youth”, the three avatars shared the virtual and real screen with many young students on the campuses of Wuhan University and Renmin University.

This is a new attempt by CCTV to introduce cutting-edge digital technologies such as virtual human technology, creating a precedent for the first immersive interactive experience of virtual programs in China.

In fact, as a national media platform, in order to create a better viewing experience, CCTV has never spared no effort in the application of new technologies.

CCTV’s party has always been an important display window for many black technologies: AR synthesis, holographic projection, 8K live broadcast, VR live broadcast, drone array, quadruped robot and many other hard technologies have all started to show common people through CCTV’s party program. popularization of life.

In a sense, CCTV’s choice of new technologies also represents the vane of cutting-edge new technologies in the cultural industry in different aspects.

The performance of the three virtual human beings in this evening conveyed an important signal to the audience very intuitively—virtual human beings, as a potential composite technology, can be naturally and seamlessly integrated into cultural and entertainment scenes.

It is worth noting that the three virtual people participating in the party this time are all from the game field.

This is not accidental in a sense, there are deeper reasons behind it –

First of all, the game industry has a deeper precipitation in virtual human technology and research and development, and can create a natural, realistic and highly interactive virtual human image through its technical strength.

Secondly, the game industry has long been committed to the creation of the digital world and the exploration of the integration of digital and real, and has a deeper and more forward-looking understanding of the application scenarios and development trends of virtual humans.

The performance of Tong Heguang, Xingtong and Gilly at the May 4th Gala is a microcosm of the virtual human technology gradually entering the life of ordinary users, and it is also an important node for the digital interaction of digital and real integration to become the norm in the future.

There is a broader value and significance behind the virtual human technology, which is worthy of discussion and analysis by practitioners in the technology industry.

Standing at the Crossroads of Technology and Humanities: The Past and Present of Virtual Humans

First, let’s get to know the virtual person briefly.

Virtual human is not a single technology, but a collection of multiple technologies.

According to the relevant definition, a virtual person refers to a virtual person with a digital appearance created through technologies such as computer graphics, graphics rendering, motion capture, deep learning, and speech synthesis.

“Hatsune Miku”, which is highly sought after by fans, is a typical virtual person. It has a beautiful character shape, a recognizable voice, and can interact and even hold live concerts.

Virtual Humans: From the Game to the Bigger World

In 1984, Max Headroom in the United Kingdom became the first virtual human to participate in the film. From this as a starting point, the virtual human began to sprout. At this time, the early virtual human was still mainly based on hand-painted technology.

Later, with the advancement of deep learning and computer vision technology, and the maturity of related technologies such as CG, speech recognition, image recognition, motion capture, real-time rendering, etc., virtual humans are increasingly moving towards the pursuit of immersion. direction.

From “Hatsune Miku” to “Luo Tianyi”, from digital Huang Renxun to virtual astronaut reporter Xiao Zheng, from monster catcher Liu Yexi to fashionista “Xing Tong”, from holographic concert to virtual human variety show, today’s virtual People have become a booming industry and play important value in various fields.

According to the qubit report, it is expected that by 2030, the scale of China’s virtual human market will reach 270.3 billion yuan, and the market demand will continue to expand.

Today’s virtual human industry stands at the crossroads of technology and humanities. On the one hand, its image creation and interactive experience are highly dependent on the technology behind it, and on the other hand, its IP and human design are highly dependent on the deep injection of humanities and emotions.

What’s under the iceberg? ——Take Xingtong as an example to see the technology behind the virtual human

If a virtual human is an iceberg, then the part we see is the part above the water surface of the iceberg, and the underlying technology that supports the virtual human is below the water surface that we cannot see.

Next, let’s take “Xingtong” as an example to take a look at the technology behind the virtual human.

The virtual human star pupil on the stage of CCTV, it was born in the game, its growth depends on the game technology, but its meaning goes beyond the game.

According to public information, Xingtong was born in 2018 and was one of the NPCs in the game QQ Xuanwu. In 2019, Tencent Interactive Entertainment upgraded Xingtong’s modeling and 3D image, and Xingtong began to move towards A wider user field of view.

Today, Xingtong is not only the virtual image spokesperson of QQ Xuanwu, but also a fashion virtual blogger who has accumulated more than 260,000 fans at station B.

Virtual Humans: From the Game to the Bigger World

Trendy styling, excellent dressing taste, blue eyes, cute face, skilled dance skills, elegant interactive conversation… Xingtong has indeed made many fans who like her shine.

Behind this is Tencent Interactive Entertainment’s unremitting attention and support in terms of resources and technology.

I have to talk about the technology behind Xingtong. The production of virtual human mainly involves three technologies: modeling, driving and rendering.

First of all, 3D modeling is the basis for building a virtual human image, and the focus is on the fine restoration of details;

Secondly, driving the virtual human model through the captured action is the main way to generate 3D virtual human actions at present, and the core technology is motion capture;

Finally, the rendering technology solves the restoration of the immersive degree of the virtual human, as well as the creation of the environment performance and effect of the virtual human.

The reason why Xingtong’s image is so vivid is that its team’s investment in virtual human live broadcast technology——

The real-time live broadcast of 3D virtual human is a collection of technologies in multiple fields. Its complexity lies in the need to integrate the real-time driving of motion capture, the real-time rendering of the engine, and the real-time synchronization of the audio and video of the live content.

This involves the precise docking of multiple coupling links such as motion capture equipment, face drive, material settlement, lighting rendering, rendering engine, live content audio and video, etc. As long as there is a small error in any link, the picture will be terrible.

In order to bite the “hard bone” of virtual human 3D real-time live broadcast, Tencent Interactive Entertainment Content Ecology Department coordinated the development resources of various internal technical teams, and created a set of real-time driving technology pipeline based on Unreal Engine4 from 0; At the same time, with Epic, FACEGOOG and other companies have strategically cooperated to achieve more difficult live broadcast effects.

After several months of repeated debugging and testing, the whole process has run through the real-time capture technology from the face, hands, and body. Through real-time skeleton calculation + Clothing Tool and other tools, the complete collection of materials such as hair, clothing, and accessories has been realized. Real-time physical motion effects.

At the same time, in order to be more realistic and natural during live broadcast, the Star Pupil technical team also used fine lighting processing + engine light chasing effect to achieve real-time rendering of multi-angle and multi-action lights .

Virtual Humans: From the Game to the Bigger World

The saturated technology research and development investment has been proved to be worthwhile. Along the way, today’s Xingtong can be regarded as the first echelon among many domestic virtual idols in terms of image expression.

Of course, in order to truly make a virtual human play a greater value, in addition to a “good-looking skin”, an “interesting soul” is also required. After all, the final destination of a virtual human is not “virtual” but “human”.

People mean personality, emotion, soul, and story, and these need to be concretely implemented from the “good-looking skin” through vivid images and warm content.

There is no interactive way that can more vividly express the personality, affinity and appeal of virtual people than live broadcast.

Since the live broadcast at station B in June last year, Xingtong has been interacting with fans almost at a weekly rhythm.

On October 23, there was a 24-hour non-stop birthday live broadcast, and on the spot, Ivan_iiivan, Bingtang IO, Qihai Nana7mi and several other well-known virtual anchors of station B were connected or interacted in the same frame. The atmosphere of the scene was very warm.

In every live broadcast, Xingtong’s cute and cute personality, outstanding dancing ability, and good conversation make this virtual fashion girl more three-dimensional and plump.

Even from the barrage of station B, careful fans can feel the subtle change of Xingtong from the slight nervousness at the beginning of the live broadcast to the ease later.

At the same time, in order to allow Xingtong to expand more possibilities, Xingtong’s operation team also let this fashionable girl break through the dimensional wall.

Xingtong has successively become “Li Ning, Levi’s Virtual Spokesperson”, “MAKE UP FOR EVER Artistic Trend Makeup Ambassador”, “Changsha’s First Intangible Cultural Tourism Promotion Ambassador”, “Inheritor of the Popular Interpretation of Peacock Dance”, “Figure Skating Champion Pang” Disciple of Qing Tong Jian”.

These labels make Xingtong a new generation of virtual idols, a vivid case of helping and nurturing diverse industries, and further broadening the application scenarios of virtual human technology.

Star pupil originated from the game, but it has already surpassed the game.

Along the way, she is the epitome of virtual human as an emerging industry boosting the integration of digital and real.

Virtual human technology does not happen overnight

If you think that Tencent’s first foray into virtual idols can quickly build Xingtong into an industry benchmark, then you are wrong.

In fact, as one of the earliest technology companies in China to deploy virtual humans, Tencent’s accumulation and breakthroughs in virtual human technology did not happen overnight—

As early as 2018, Siren, a high-fidelity virtual human co-created by NExT Studios, was driven by real-time expressions, which caused a sensation in the industry at that time.

Matt, born a year later, has been able to achieve voice-driven facial expressions through AI technology, which closely associates voice with vivid facial expressions and emotions.

In 2021, based on real-time high-fidelity virtual human technology, Tencent NExT Studios and Xinhua News Agency jointly launched China’s first digital reporter and the world’s first digital astronaut Xiao Zheng, and completed several aerospace reports.

And this time, Tong Heguang and Gilly, who appeared on CCTV on the same stage with Xingtong, also have their own characteristics——

Gilly uses Unreal Engine’s next-generation real character model generation system MetaHuman and the cutting-edge Facegood face capture technology to make limbs and expressions more vivid and natural;

Behind Tong Heguang are the xFaceBuilder® digital character production and xMoCap® motion capture animation full-process production pipelines developed by NExT Studios and the technical support of the two laboratories of xLab photo modeling and motion capture. They are also representatives of male virtual idols.

Virtual Humans: From the Game to the Bigger World

It can be seen that the accumulation and breakthrough of virtual human technology is inseparable from the forward-looking layout and R&D accumulation of game companies. In essence, the game technology has developed to the current stage and has the ability to ‘go out’, which can empower more upstream and downstream industries.

The virtual human technology, born out of games, has developed into a “new species” that cross-covers computer graphics, artificial intelligence, virtual reality and other frontier disciplines of the Internet, and has an extremely wide range of applications in many fields.

The application of virtual human – bearing fruit in multiple industries

The virtual human industry has not stopped at the sub-track of virtual idols. Today, it has already moved towards a wider range of application scenarios.

Generally speaking, virtual people can be divided into two categories: identity type and functional type:

Identity-type virtual people are mostly virtual IPs and idols, with obvious personality and IP attributes, while functional virtual people are mainly used to replace real people to serve, and are virtual representatives of service-type roles in the real world.

Today, these two types, as the two routes of the virtual human industry, are accelerating out of their own characteristics——

First of all, let’s look at the identity-based virtual idol. One of the important reasons for its development is that compared with real idols, virtual idols will never have the risk of “collapse”. Therefore, major entertainment companies are also actively deploying virtual idols market.

Looking at functional virtual human, its development stems from its practical value, such as the simplest scene – the museum docent, which can effectively save the labor cost of the museum.

With the advancement of technology, the production cost of virtual people has gradually decreased, so new formats such as digital employees and virtual anchors have been developed one after another, and their applications in entertainment, e-commerce, education, cultural tourism and many other industries have also begun to gradually land.

For example, Hunan Satellite TV launched the virtual host Xiao Yang in the “Hello, Saturday” program that premiered on January 1, 2022.

The support of the state at the policy level is also an important driving force for the vigorous development of functional virtual humans.

In October 2022, the “14th Five-Year” Science and Technology Development Plan for Radio, Television and Network Audio-visualization pointed out that virtual anchors and animated sign language should be widely used in news broadcasts, weather forecasts, variety shows, science and education programs.

This points out the direction for the development of functional virtual human from the policy level.

It can be seen that with the increasing development of the demand side and the supply side, both identity-based virtual humans and functional virtual humans have shown considerable potential in their respective fields.

The Game Technology Behind Virtual Humans: A “Dream Workshop” for Building a Bigger World

The virtual human technology is born out of the game technology, and it keeps making breakthroughs with the progress of the game technology.

We can intuitively feel it through the following picture-

Virtual Humans: From the Game to the Bigger World

This is the evolution of Lara Croft in Tomb Raider.

It can be clearly seen that with the advancement of game technology represented by deep learning, real-time rendering, CG, motion capture, and machine vision, the image of virtual human Laura has changed from blurred to clear, from clumsy to smart, from cartoon to real. .

The vividness and fullness of the virtual human image in turn further enhances the game experience.

In the macroscopic “super digital scene” of the next-generation Internet, game technology and the virtual human technology derived from it will, in a sense, become the “meta-technology” in the future digital-real integration industry, and will play a role in “dream maker”. The infrastructural role of the workshop.

The core logic behind this is that the next computing platform will definitely blur the boundaries between “reality” and “virtual”, whether it is based on the relevant blueprint of the Metaverse or the relevant definition of the next-generation true Internet.

In this process, “identity” is an extremely important part.

What’s the meaning?

In today’s online world, what defines our identity is a simple avatar in WeChat, a cartoon hero in Glory of Kings, and a half-length figure without legs in Facebook’s VR social scene. Very singular, very “unvivified”.

The most essential reason behind this is that the bandwidth of these “Avatar” inputs is too small, and it cannot intuitively reflect our “spirit” including actions, demeanor, and emotions.

In the future, this input bandwidth will definitely increase. At that time, as human beings in the real world, we can complete the real “onlineization”, which actually belongs to the generalized virtual human technology.

And no matter how the next generation computing platform changes, the two factors of “people” and “world” are very critical.

These people include us and our digital avatars, others and “other people’s digital avatars”, and fully digital “pure virtual people”.

The world includes not only the real world, but also the virtual world, and the world where the digital and real are integrated.

In order to realize the mapping and extension of the real world, the virtual human technology “spillover” from game technology can create virtual people and “real people clones” that are indistinguishable from real people, and the game technology itself can be constructed from 0 with high efficiency and high quality. A digital scene with shape, sound, and color.

Don’t think that the future I describe is very far away, in fact, we can already see that game technology not only “spillover” to the virtual human industry, but also “spillover” to more industries and scenarios——

For example, game technology is widely used in the film and television production industry, which can effectively improve the flexibility and expressiveness of film and television creation, while reducing the cost of shooting.

During the filming of the popular American drama “The Mandalorian” in 2019, the filming team used Unreal Engine4 to create detailed Star Wars scenes for digital virtual shooting.

The automotive industry has also benefited from technologies born out of games . For example, game engines, as real-time 3D creation platforms, can be applied to segments such as automotive industry design, simulation training, and autonomous driving.

For example, in terms of autonomous driving, Tencent TAD Sim introduced UE4 to simulate weather and traffic conditions under real driving conditions with the help of a physics engine, which can achieve environmental simulation effects.

Game engine-driven digital twin systems can also help smart city planning. For example, Hong Kong Airport has developed an airport operation and maintenance system based on digital twins.

When John Carmack, the originator of game engines, developed the DOOM engine and announced that it would be open sourced in 1997, he would never have imagined that the technology he created would surpass games decades later and be so widely used in film and television, Automotive, automation and other industrial fields.

Director Yang Dechang said that since the invention of the film, human life has been extended by at least three times.

In a sense, after the invention of games, human life at least tripled on the basis of movies.

What connects the “game world” and the “real world” is the technology behind the ninth art game that constantly “turns corruption into magic”.

The game of that time will no longer be a game.

Posted by:CoinYuppie,Reprinted with attribution to:
Coinyuppie is an open information publishing platform, all information provided is not related to the views and positions of coinyuppie, and does not constitute any investment and financial advice. Users are expected to carefully screen and prevent risks.

Like (0)
Donate Buy me a coffee Buy me a coffee
Previous 2022-05-06 23:36
Next 2022-05-06 23:42

Related articles