Low Latency Hand Tracking & Auto Power To Update

The Meta Quest v56 update brings a range of improvements and new features to Quest 2 and Quest Pro.

Version 56 seems to be the most feature packed update in quite a while. As always, remember that Meta “rolls out” system updates gradually over the course of weeks.

Hand Tracking 2.2

Since releasing controller-free hand tracking as a software update for the original Oculus Quest in late 2019, Meta has been steadily improving the technology with further updates.

The 2.0 update last year improved handling of fast movements, hands occluding each other, and hands touching together – three scenarios which frequently broke the original algorithm. Then the 2.1 update in January reduced tracking loss and sped up reacquisition when tracking is lost.

With v56 Meta is launching Hand Tracking 2.2 which it says reduces hand tracking latency, “bringing the experience closer to controllers”. Meta is claiming up to 40% latency reduction in typical usage and up to 75% during fast movement

Hand Tracking 2.2 also introduces a new Fast Motion Mode (FMM), which Meta says improves tracking of “fast movements common in fitness and rhythm apps”, though it doesn’t mention what tradeoffs this entails.

To showcase how far hand tracking has come on Quest, Meta is releasing a demo app called Move Fast, showing how it can be used even for fast paced scenarios.

Auto Power to Update

Arguably the worst part of the standalone VR experience is waiting for the app you want to play to update. Unlike other mediums, you’re essentially blindfolded and can’t easily use your phone, eat, or drink. Worse, there’s no indication outside the headset of an update’s progress so you just have to keep checking.

While Meta recently updated Quest to automatically update apps when charging and added an “Update before powering off” toggle to the Power off interface, there’s still a remaining scenario where apps and the system wouldn’t get updated: if your headset battery reached 0% during play.

Plugging in your headset would have it charge, but since it wouldn’t be turned on it wouldn’t update. But v56 adds an option to change this. When you plug in your headset it will automatically turn on and download both app and system updates.

Version 56 also shows your battery level on the power off screen. Meta’s release notes also mention additional, unspecified, improvements to the low battery warning notification.

System-Wide Local Dimming On Quest Pro

Most LCD panels have at most a few LEDs backlighting the entire display, severely limiting the contrast between dark and bright regions. You’ve probably noticed they show dark gray instead of true black. In VR this notably hampers the visual believability of virtual nighttime and experiences set in space.

The main advantage of OLED (eg. PlayStation VR2 and Bigscreen Beyond) over LCD is there are no backlights – each pixel is self-illuminating, enabling effectively unlimited contrast and true blacks.

LCDs with local dimming, such as the Quest Pro panels, offer a middle-ground solution. They have many backlights to control brightness in zones – in Quest Pro’s case over 500 mini LEDs.

However, local dimming on Quest Pro is opt-in per-app. App developers have to enable it. Otherwise, all 500+ backlights will act together with the same brightness as if a regular LCD.

With Version 56, Meta is adding the option to enable Local Dimming system-wide, except for apps where developers have specifically opted out.

Keep in mind that local dimming has a noticeable tradeoff though: blooming. Since the backlighting resolution is so much lower than the color resolution – on Quest Pro just 500 mini LEDs for almost 4 million pixels – many dark-colored pixels adjacent to light ones will also be illuminated. Mini LED is a big step over traditional LCD, but is not a full substitute for OLED.

Facebook Livestreaming Is Back

Both the original Oculus Quest and Quest 2 shipped with support for livestreaming to Facebook, but Meta removed the feature in 2021, citing low usage relative to local casting.

Two years later, Meta claims “by popular demand”, the feature is coming back as an experimental update that will “roll out gradually” separately to the main v56 update.

Unlike the previous implementation, the new feature will show a floating persistent panel with the current viewer count and live chat. Could Meta eventually add support for native livestreaming from Quest to other streaming platforms?

Live Captions In Home & Button Remapping

Version 56 also brings two new optional accessibility features.

Live Captions will show floating subtitles for any playing audio in Home apps like Quest TV, the Explore tab, and the Quest Store. But the company says it’s exploring the possibility of expanding this to other apps – potentially system-wide.

Button Remapping is the other accessibility-focused feature. Meta says you’ll be able to swap buttons, create custom layouts, and more.

Swipe Typing On Virtual Keyboard

Version 56 also adds Swipe Typing for the virtual keyboard as an experimental feature, as spotted by some Public Test Channel testers last week.

Meta Quest v56 adds an Experimental Feature for Swipe Typing on the Direct Touch keyboard. pic.twitter.com/UwhD80cTsm

— Luna (@Lunayian) July 11, 2023

Swipe typing is an input method available on smartphones, and some people find it allows faster typing than individually pressing keys.

Swipe typing could be uniquely suited for VR and AR where virtual keyboards have no physical haptic resistance.