I just experienced next-level leg tracking and mixed reality with Pico 4

Leg tracking for the Pico 4

Using the controllers and the quartet of 6DoF inside-out cameras, the Pico 4 tracks your hands among other things, which is nothing new so far. But the eternal question remains: What about the legs? We were able to check out pre-production units of Pico's new Leg Trackers at MWC.

Neither the market launch nor the name of the leg trackers has been determined yet, but the design has largely been finalized, apart from the large stickers, of course. At least that is what the manufacturer informed us.

The principle is simple: You strap a tracker around your right and left leg, and voilà! Your digital self has legs. This is not only great for avatars but also offers exciting avenues for new applications. At MWC, we tried a fitness app where you had to chop blocks in "Beat Saber" style using one hand, as well as jump onto approaching blocks using your legs.

Thus, the game not only requires a new dimension of body control but also becomes really sweaty after a while. VR or MR definitely offers enormous potential when it comes to fitness applications. According to a Pico spokesperson, the leg trackers could also be attached to the arms, for instance, to further improve accuracy.

• Also read: How does the Meta Quest Pro perform in the hands-on?

Improved RGB passthrough for MR applications

The Pico 4 can also expect a revised RGB passthrough mode within the next three to six months. Here, the VR goggles will mix the image captured via the integrated RGB camera with virtual content. During the MWC demo, the demo room was mostly visible. While this was previously only possible in "2D" and at a poor quality at that, a noticeable revision took place in our test unit. A wall was virtually "broken open" and monsters and dragons came flying into the room.

The biggest challenge for the Pico 4 here is that it only has one 16 MP RGB camera available on the front for the passthrough mode, which sits right between the user's eyes. This solitary camera only 'sees' in 2D. So, the Pico 4 relies on algorithms to calculate depth into the image information and seamlessly combine the camera image with virtual content.

For things that are located further away, this actually works very well. However, spatial perception is noticeably limited when it comes to close objects. This is noticeable in such instances when you want to grab things from reality based on the passthrough image. In such an environment, the hands tend to grab nothing as it takes some getting used to.

This worked much better with the (and admittedly, three times more expensive) HTC Vive XR Elite in our first hands-on. Not only is the depth perception excellent thanks to the built-in dedicated depth camera, but the image quality is also much better. Printed text is still difficult to read via the Pico 4's passthrough feature.

With the HTC Vive XR Elite in particular, I could easily grab my phone from the table and write as well as read WhatsApp messages. Do take note that the revised MR feature upgraded the Pico 4 enormously in my eyes, which is a real plus point.

What about the third new feature for the Pico 4?

In the coming months, the Pico 4 will receive another feature that is particularly exciting for content creators. With the help of the smartphone app, it will be possible to determine the precise positional relationship between the smartphone and the VR user. This will, in turn, enable a mixed reality capture function. If you film yourself with your smartphone, the app will then transport you to the virtual world you are currently roaming in the video.

This is supposed to work best with a green screen (chroma keying), although the Pico spokesperson stressed in an interview with NextPit that the green screen is optional. The previously mentioned algorithms for distinguishing foreground and background are supposed to work precisely enough to detach the player from his gaming environment.

• Also interesting: The best standalone VR glasses

What do you think about the new features for the Pico 4? Do you think the inclusion of the passthrough function is long overdue? Or are you not into MR applications at all? Do you imagine doing a fitness workout in virtual reality? I look forward to your opinion in the comments!