The 5 major equipment that enters the Metaverse can be seen and touched

As we all know, Facebook has been renamed Meta and is committed to building a meta-universe. In order to realize its grand vision, it will inevitably need to launch multiple breakthrough devices, controllers and hardware support.

Meta has shown some concepts and research projects in the field of meta universe, some of which are quite fascinating. These strengthen our connection with the virtual world to a certain extent, and at the same time improve our ability and productivity to perform operations in the meta-universe.

The following is the full content of the current Meta universe equipment strategy. Let’s take a look~

Reality Labs Tactile Gloves

Reality Labs haptic gloves are dedicated to people’s tactile interaction in the virtual world. When we put on gloves and move in concert, we will “feel” grasping objects, textured surfaces, and even the weight and softness of objects in the virtual world.

The 5 major equipment that enters the Metaverse can be seen and touched

A newly designed soft actuator is placed in the glove, and there are soft motors everywhere on the glove. The soft brake moves together with the soft motor, which can bring a good tactile experience to the glove wearer. The glove also has a microfluidic chip that moves the actuator by controlling the airflow.

From a technical point of view, when we pick up a feather or metal ball in the virtual world, there will be a different “feel”. Zuckerberg has already experienced wearing tactile gloves in virtual reality (VR).

A series of virtual interactions of Reality Labs haptic gloves. These include manipulating virtual objects, such as virtual ball throwing and catching; multi-person interaction, such as thumb wars, handshake; and multi-player games, such as virtual building blocks. (Video: Facebook/Tech@Facebook)

“The value of both hands is huge in solving the interaction problems in AR and VR,” said Sean Keller, director of research at Reality Labs.

Imagine doing virtual puzzles with your friends’ super-realistic 3D avatars. When you pick up a piece of virtual puzzle from the table, you can feel it in your hand and your fingers will automatically stop moving. When you pick up the cardboard for closer inspection, you can feel the sharpness of the edges of the cardboard, the smoothness of the surface, and then you will make a clicking sound when you manipulate the cardboard.

Or sit down and work in a cafe, and a virtual screen and keyboard appear in front of you. The virtual keyboard fits the hand shape and available space, and can be personalized to meet your various preferences. You can feel the click sensation of each keystroke, as well as the edges of the virtual keys on your fingertips, typing as easily as on a real keyboard.

Meta said that the glove technology has been developed for more than seven years, but it is currently unable to be commercially released.

The challenge has just begun. It is currently impossible to build a mass-produced, affordable consumer device that allows people to experience any tangible interface anywhere. A truly new type of tactile glove requires breakthrough development across many scientific and engineering disciplines.

All-in-one VR headset Project Cambria

VR headset (virtual reality helmet) is one of the key devices to enter the meta-universe. Meta announced Project Cambria, the next-generation all-in-one VR headset that will be unveiled next year.

At the Connect conference at the end of October, Meta stated that Project Cambria is neither a replacement for Quest 2 nor Quest 3. It is a higher-priced high-end device that will use a series of advanced technologies, including optimization of social presence. Sense, color perspective, Pancake optics, etc.

This device will focus on mixed reality applications (that is, augmented reality (AR) and VR are used together, controlled by joystick or manually), the device will include real-time face tracking and eye tracking functions, we can express ourselves in the meta universe mood.

Fortunately, this product does not have to wait too long.

Electronic tactile skin ReSkin

Existing tactile glove solutions are difficult to scale, and are still limited by factors such as cost, durability, use threshold, volume, and weight. In order to solve these problems, Meta’s AI department collaborated with scientists at Carnegie Mellon University to develop a tactile sensor called ReSkin (only 2-3 mm thick), which can also be understood as a flexible plastic “skin” .

ReSkin is a flexible tactile sensing solution that uses a deformable elastic material with built-in magnetic particles, so when it deforms, the electromagnetic signal it emits to the surrounding will also change. Then, you can use a magnetometer to approach ReSkin to measure the electromagnetic signal changes, and use algorithms to convert the data into usable information, such as: contact location, strength, and so on.

The 5 major equipment that enters the Metaverse can be seen and touched

Using soft fruits such as grapes and blueberries to compare the difference between using ReSkin (top) and using the built-in force sensor (bottom), it is found that the use of ReSkin magnetic signals can detect forces as low as 0.1 Newton from a 1mm wide object. It can effectively avoid object deformation caused by excessive grip.

The 5 major equipment that enters the Metaverse can be seen and touched

Connecting the 8 sensor modules of ReSkin together, adding sensors can increase the tracking area and identify the pressure exerted by the finger on any area of ​​the sensor. In other words, the ReSkin sensor can be used to cover the entire glove in the future to improve the tactile perception ability of robots or somatosensory gloves.

The 5 major equipment that enters the Metaverse can be seen and touched

Integrate ReSkin on the glove to track the changes in the strength of the hand bag, and the subtle tactile effect is good enough

Meta said that this new material is expected to support the development of Metaverse.

ReSkin is low cost, light weight, small size, compatible with any holder, and can be used immediately. It is reported that the cost of manufacturing one hundred pieces is less than $6 per piece, and the higher the output, the lower the cost. It is also durable enough to support more than 50,000 interactions, and the sampling rate is up to 400Hz, the error is only 1mm, and the accuracy rate is 90%. After a certain degree of wear, it can be easily replaced like a bandage.

This makes ReSkin an ideal material for robotic hands, tactile gloves, and pet insoles. In other words, these devices equipped with ReSkin can help researchers use tactile data to train AI models. Before that, it was difficult or even impossible to collect relevant data.

Wristband AR sensor

Every technology company wants to have awesome AR glasses. In addition to glasses, you also need a sensor to interact with objects in AR.

In March of this year, Meta launched a wristband AR sensor that can detect hand movements. Through electromyography (EMG), the device can detect electrical signals generated by wrist and finger movements and convert them into commands at the same time. You can use these commands to control different devices.

The 5 major equipment that enters the Metaverse can be seen and touched

The company claims that the signal can be detected even if the finger moves one millimeter. In the future, we can even feel the intention of finger movement.

(The intention of finger movement is not mind reading. Think of it this way: when you take a lot of photos and choose to share only some of them. Or you have a lot of ideas and you only act on some of them. When this happens, The brain will send signals to your hands and fingers, telling the fingers to move to the place you choose in order to perform actions such as typing or sliding.)

Initially, this technology can only detect gestures such as pinching and releasing the finger, and a wealth of control functions will be added in the future, allowing us to touch and move virtual objects (UI) in AR, which is a bit like super powers. .

In addition, the research team is still further exploring, trying to develop the function of high-speed typing on the desk or lap. The virtual keyboard can not only be used at any time, but also can learn and adapt to each person’s different typing styles over time, and produce Adaptability changes and even faster typing speeds are achieved.

The FRL team has made important progress in the personalization model, which reduces the training time of the keyboard model adapted to the personal style. (Video: Tech@Facebook官网)

The FRL team is currently working on exploring adaptive interfaces that adapt to specific environments to achieve “smart clicks.” There is no need for the user to move the line of sight or browse the menu. The system actively provides an adaptive interface by predicting the user’s intention in different environments, and the user only needs lightweight feedback to control it.

We only need to use some micro gestures, no need to stop what we are doing, and no need to be distracted to save time.

The 5 major equipment that enters the Metaverse can be seen and touched

Virtual bow and arrow. Under the action of the wrist sensor, you can feel the sensation of pulling the bowstring, making you sure that you are shooting an arrow and judging whether you are doing the right action.

You can also customize your settings. When you receive an emergency email, the bracelet can vibrate more strongly than when you receive a regular email. When there is an incoming call, you can know the identity of the other party through different tactile feedback. . The researchers even added: “Touch can also convey different emotions, which may be a new way to improve social communication.”

The realization of these functions requires the application of technologies such as machine learning, human-computer interaction, user interface design, etc. There are still many technical challenges. Although the use of ultra-low friction input methods can achieve interaction with the adaptive interface, if you want to perceive virtual objects, you need to continue to develop tactile technology.

Currently, the Facebook team is building research prototypes, including Bellband with 8 pneumatic bellows, and Tasbi with 6 vibrating actuators and a novel wrist squeeze mechanism to further explore the types of tactile feedback.

Project Aria AR glasses

Meta wants to fight for AR dominance, but its first concept project is still quite “docile”. Last year, it launched augmented reality (AR) glasses code-named Project Aria. This year, we learned more about this product mainly due to its product report exposed in August.

The 5 major equipment that enters the Metaverse can be seen and touched

The 5 major equipment that enters the Metaverse can be seen and touched

On the sensor side, it has a camera and three buttons for power, capture and mute. Project Aria devices are not conspicuous in daily life, similar to ordinary glasses, but they have platforms and sensors to capture the user’s video, audio, location and eye tracking. But at present, it is only a competitive product of Snap Spectacles.

In short, Facebook once again emphasized in the product manual that this AR glasses is not currently a commercial product. It will initially be used by scientific researchers for testing and data collection to help Facebook explore future wearable AR glasses that may need to be equipped with sensors and other designs.

What’s next

The above Project Cambria earphones may be on the market next year, but things like tactile gloves may take many years to be on the market. In addition, Facebook invested 10 billion US dollars earlier this year to develop Metaverse hardware and software. Therefore, if we see a surprising product soon, please don’t be surprised.

Posted by:CoinYuppie,Reprinted with attribution to:
Coinyuppie is an open information publishing platform, all information provided is not related to the views and positions of coinyuppie, and does not constitute any investment and financial advice. Users are expected to carefully screen and prevent risks.

Like (0)
Donate Buy me a coffee Buy me a coffee
Previous 2021-12-01 07:34
Next 2021-12-01 07:36

Related articles