We saw a lot of realityOS leaks during WWDC 2022 - but we don’t know it yet. All enhancements seem solid build-out toward a conceptual MR environment. Here are a few examples.

Leaks

Is it was reported by Techspot and Macrumors, there are number of references to RealityOS in Apple’s source code, this is giving a community a clear signal that both OS and MR device are coming to the market, and only release date is unknown.

“Apple's upcoming mixed reality headset will be driven by a custom operating system known as realityOS, developers have discovered. The naming scheme makes sense, at least on paper, as Apple has given similar names to its other operating systems including iPadOS and watchOS.”

Mentioned source code is still available on github

Features

• StageManager jumps out from the screenshot below as something similar to what Meta has in their Oculus ecosystem. Bold, floating windows, the side docked window design seems perfect for peripheral vision, and it seems there is no need for a new window management system in macOS - this could be preparation for future products.

• Generally, UI Elements - e.g. bold text, overlays, key features to make content visible in a noisy mixed reality overlay.

• The Lock Screen is revised from time to time, and this year is no exception. There is a lot of work around occlusion, fonts and design that lends itself to RealityOS. Apple's Newsroom post says: “Notifications have been redesigned to roll up from the bottom, ensuring that users have a clear view of their personalized Lock Screen” A clear view of what? - Your MR headset Viewport.

• App Intents session recommended that developers are accommodate audio-only environments in specific ways. The new live designs for intents also seem really relevant.

• CatPlay dashboard looks like a proof of concept for UI they’re planning. Apple is known to experiment with platforms to prove concepts for others devices.

• Live Text looks like a perfect app for a MR Headset, Live translation is an obvious feature you would add to a revolutionary headset device.

Frameworks

Apple has unveiled some new framework features that seem to form the basis of OS. Taken out of context, they might raise the question of why these investments are worthwhile. However, when viewed from the perspective of AR, this becomes clear. Here are some examples:

• Charts and CalendarUI engineers have been developing open source libraries since iOS 4. There should be a reason why they have developed these frameworks now, probably to fulfil an internal product or platform need.

• WeatherKit has nice benefit around consistency. If it’s raining, you are 100% sure that all apps and system know it’s raining and goes into rain mode.

• LIDAR has been touted as a new feature on iPad and iPhone for the past few releases. You know how Oculus asks users to onboard by defining a virtual safe zone to avoid bumping into things? LIDAR - is a major player in their RoomPlan functionality, and potentially a key capability for Mixed Reality as well.

One more thing

Look at how Apple is adding more Mixed Reality foundational features to it’s products - Apple Watch Assistive Touch feature, just have a look and imagine how this could enable input system for Mixed Reality headset.