This is a review of VisionOS itself, not the Apple Vision Pro. I’ll occasionally touch on hardware briefly since it can have a substantial impact on the execution of VisionOS. However the primary focus is on interactions with Vision Pro regardless of the hardware, such as hypothetically being able to use it without any device at all.
I’m actually shocked at just how basic it all is. For being hyped as a new era of computing, the years of rumored development time, and the fact that it is clearly derived from a very mature iPad OS, it feels like VisionOS was put together last-minute. It’s like they just went with the first idea that came to mind rather than brainstorming and iterating through several ideas. An example of this is the control center that leans way too heavily into the “pull down from the top” paradigm. While that works great for phones, it really makes no sense in a “spatial” context, and it shows in VisionOS.
I call it perplexing complexity because VisionOS is characterized by high-level concepts (with corresponding impressive technical feats) that are hamstrung by poor exectution. For example, it has iris scanning, but can’t detect multiple users? If has best-in-class hand tracking but you can’t move app icons? I only used it for 2 weeks and had several ideas about it could be better - why was this the best they could do?
Overall VisionOS Concept
In short, VisionOS is about using 2D apps in the space around you. I‘m completely sold on this idea as the future of computing. There’s a lot of avenues they could have chosen, and by comparison, this approach seems the most basic, perhaps even boring. But I actually think it’s the most appealing one. For context for the following, consider reading the Verge’s Apple Vision Pro Review where it defines AR, MR, and VR
A predominantly AR experience works best when it’s passive. Instead of constantly querying, “show me this” or diving into apps, the system intelligently surfaces information as it’s relevant. However, you would need integration with 3rd parties, and I don’t see how this wouldn’t devolve into a hellscape of digital billboards. It’s also hard to imagine compelling use-cases outside of a few niche uses.
The endgame of a predominantly VR experience is the metaverse. A second-tier reality of social media on steroids is not what society should be aspiring for. Tuning out to enter a virtual reality works well for gaming, but it’s hard to see a compelling use case beyond that.
They could have gone with a predominantly 3D mixed reality experience. The problem is that 3D is hard. I got my degrees in engineering and can count the number of times we solved problems in 3 dimensions on one hand. It’s hard to create 3D content; 2D UI’s are hard enough to get right, let alone an effective 3D UI. It’s also hard to consume 3D content. You can’t just grab it, so you have to resort to unnatural gestures to manipulate objects. It is not a great user experience.
In contrast, 2D apps have proven utility. They are simple enough abstractions to be easy to use, but powerful enough for a high-degree of capability. You can equally have a simple messaging app and a powerful photo editor within the same paradigm. The idea that you can spin up windows on-the-fly in the space around you is very compelling.
The “Spatial Computing” Contradiction
Even though the use case is compelling, Apple did not execute it well. You have probably read comments about VisionOS needing better window management. The problem is deeper than that.
![1708658628491.jpeg]( "1708658628491.jpeg")
As shown above (from Apple) VisionOS meticulously maps out the environment with LiDAR, cameras, motion sensors, etc. But… why? You can’t align windows to the environment, such as flat against a table, flush against a wall, or aligned orthogonally with your desk. If you want to, you know, put apps in your space, you have to be meticulous. Side-step around for every app, face just the right way, and finely move your fingers until it’s in position. Yet even then it won’t be quite right, like a crooked picture frame.
Unfortunately you can’t align apps with each other, either. Apps launch at some pre-determined distance such that they are facing perpendicular to whichever way you happen to be gazing at that moment. If another app is close enough, it will launch in front of it, with the other one fading out. You can move apps laterally, which causes them to rotate to continue facing the position you happen to be in. You can move apps forwards and backwards, which causes them to proportionately grow/shrink. They don’t actually change in angular size like in the optical illusion below (every image of the car is the same).
![1708660008975.png]( "1708660008975.png")
With this system, there’s no point in having such flexibility to position apps. It’s basically impossible to line apps up at the same size, uniformly spaced laterally, and at the same depth. Any apps that overlap in terms of line-of-sight (regardless of their position/depth) will cause one to fade out as a portion of one window goes transparent. You have to alternate which one is active/focused by looking at tapping the desired window.
This is all to say that aligning windows is deeply unsatisfying. It’s really the worst of both worlds. It’s not a window-management system designed to be integrated into the environment. But it’s also not designed for an exclusively virtual-positioning system. Instead of just a haphazard collection of randomly sized windows that are just kind of “there”.
Additionally, the scale of app content is just too small. The minimum size of a window (even all the small setting) is probably about the size of a 16” screen, yet shows less content as Than the 12.9” iPad. The novelty of 4’ screens is not so great when they need to be that big to be useful.
Virtual Monitors, Not Windows
David Pierce wrote a great article prior to the Apple Vision Pro unveiling about the potential of the upcoming platform. It can be summed up as if you see a homescreen, they blew it. Well, we got a homescreen.
It’s unweidly having several app windows floating around. They tend to occupy a huge portion of your FOV; 3 windows tends to be a practical limit of working with VisionOS. Turning your head frequently is annoying and prone to repetitive stress. While it’s great to be able to spin up multiple “monitors” whenever you like, it’s silly to need one for every app. Imagine if you had to plug in another monitor for every app you wanted to use on MacOS. It’s not a solution that scales well.
Instead, VisionOS shouldn‘t be launching apps within our space from a homescreen. It should be launching virtual monitors within our space, which can host apps. I’m not talking about MacOS virtual desktops, screen casting, or display mirroring. I’m talking about a simple UI scheme that effectively reserves a plane of space for an app, then lets you stack apps in the same spot. You launch multiple VisionOS apps within the monitor along with a dock to swap between them at the same position. You could have one floating window to work with all your apps if you want. Then at any time you could pull out apps into 2, 3, 4 monitors - whatever you like.
Environments
Environments are the best thing about VisionOS. I love how they are photorealistic rather than 3D generated like a video game. Nevertheless, the execution leaves a lot to be desired.
For one, they are basically apps that launch from the homescreen (also in whichever direction you happen to be gazing at the time). While beautiful and legitimately superior to pass-through video, it’s VR. It means that you suddenly lose all context of your surroundings - meaning you lose the “spatial” in spatial computing. You know all that environ mapping VisionOS is doing? Yeah kinda pointless within an environment. You can only see your app windows floating ambiguously in the nether and your hands (that are poorly masked). As a half-assed compromise, the dial controls the degree of “immersion” so that you can somewhat enjoy an environment while the concept of VisionOS still makes sense.
Instead, environments should open up in a window just like the encounter dinosaurs app. The dinosaur app detects a nearby wall (if available) and opens flush against it, as if the wall no longer exists but is instead a portal to another world. I’d like environments to act similarly - be able to replace a wall with a view into the environment. That way I can work normally at a desk with my virtual monitors, but when I look to the left I see a serene location rather than a view into a parking lot.
Input and Control Center
The eye and hand tracking make for a great combination. I think it was a great choice to go with a solution that did not utilize controllers, but it’s not without its downsides. It does limit the usefulness of VR experiences such as games. Other platforms use controllers to let users to move around a virtual environment (preventing users from bumping into furniture and walls). VisionOS has no such affordance, so any VR experiences are going to have users staying put. Nevertheless, I think they are on the superior side of that equation.
That being said, hand-tracking is really basic and an aspect that makes VisionOS feel oddly rushed. Why is there only a pinch? There’s not really any gestures in the way that iOS has gestures. It’s all just a way-too-literal translation of replicating touchscreen gestures when there is no touch screen. Swiping on a touchscreen on iOS is intuitive a way that pinching-and-flicking in mid-air simply isn’t. I feel like they should have put more thought into gestures that are more natural in context.
Control center is another example of something taken much too literally from iOS. Look up to expose a hidden button that can be tapped? It’s unreliable and its behavior totally inconsistent with open apps. Typically you have to look up with your eyes, but when there’s an app you also need to look well above the app. Yet I’ve also had it appear over videos and panorama photos. Snazzy Labs made the comment about why isn’t something like a a button that appears on your wrist, given the fancy hand-tracking and all. Perplexing complexity
Hampered by Hardware
Although not a review of the Apple Vision Pro, the reality is that VisionOS can only exist through hardware. Right now, it noticeably degrades what VisionOS wants to be. The pass-through video is not high quality, meaning it’s not paritcularly convincing that these virtual apps exist in my reality like magic. Tim says that that Vision Pro “seamlessly blends” virtual and physical object. Well, there are seams, and they are quite noticeable. It’s not so much that it feels like your 2D apps are in the space around you. It’s more that you are keenly aware that you need to be looking through a large contraption to see them.
The eye and hand tracking for VisionOS means that any hardware for it needs to be plastered in all kinds of sensors, cameras, and compute power. Anything that runs it is going to need to be beefy contraption for a long time.
Conclusion
In broad strokes, I think Apple made all the right choices for VisionOS. Positioning it as a platform for a tried-and-true paradigm of 2D apps with controller-less interaction was a great choice. Unfortunately, they fumbled the execution.
More than any other proposal for the future of computing, I’m actually excited by what VisionOS could be. The ability to pull-up virtual, multi-monitors at will on my desk, virtual TV’s on my wall, and virtual windows looking over the ocean.
As-is, however, it’s not all that intriguing - even if I could see it without any device at all. It would be kind of a mediocre experience of siloed windows of huge content, floating just kind of wherever, all getting in each others’ way.
I’m actually shocked at just how basic it all is. For being hyped as a new era of computing, the years of rumored development time, and the fact that it is clearly derived from a very mature iPad OS, it feels like VisionOS was put together last-minute. It’s like they just went with the first idea that came to mind rather than brainstorming and iterating through several ideas. An example of this is the control center that leans way too heavily into the “pull down from the top” paradigm. While that works great for phones, it really makes no sense in a “spatial” context, and it shows in VisionOS.
I call it perplexing complexity because VisionOS is characterized by high-level concepts (with corresponding impressive technical feats) that are hamstrung by poor exectution. For example, it has iris scanning, but can’t detect multiple users? If has best-in-class hand tracking but you can’t move app icons? I only used it for 2 weeks and had several ideas about it could be better - why was this the best they could do?
Overall VisionOS Concept
In short, VisionOS is about using 2D apps in the space around you. I‘m completely sold on this idea as the future of computing. There’s a lot of avenues they could have chosen, and by comparison, this approach seems the most basic, perhaps even boring. But I actually think it’s the most appealing one. For context for the following, consider reading the Verge’s Apple Vision Pro Review where it defines AR, MR, and VR
A predominantly AR experience works best when it’s passive. Instead of constantly querying, “show me this” or diving into apps, the system intelligently surfaces information as it’s relevant. However, you would need integration with 3rd parties, and I don’t see how this wouldn’t devolve into a hellscape of digital billboards. It’s also hard to imagine compelling use-cases outside of a few niche uses.
The endgame of a predominantly VR experience is the metaverse. A second-tier reality of social media on steroids is not what society should be aspiring for. Tuning out to enter a virtual reality works well for gaming, but it’s hard to see a compelling use case beyond that.
They could have gone with a predominantly 3D mixed reality experience. The problem is that 3D is hard. I got my degrees in engineering and can count the number of times we solved problems in 3 dimensions on one hand. It’s hard to create 3D content; 2D UI’s are hard enough to get right, let alone an effective 3D UI. It’s also hard to consume 3D content. You can’t just grab it, so you have to resort to unnatural gestures to manipulate objects. It is not a great user experience.
In contrast, 2D apps have proven utility. They are simple enough abstractions to be easy to use, but powerful enough for a high-degree of capability. You can equally have a simple messaging app and a powerful photo editor within the same paradigm. The idea that you can spin up windows on-the-fly in the space around you is very compelling.
The “Spatial Computing” Contradiction
Even though the use case is compelling, Apple did not execute it well. You have probably read comments about VisionOS needing better window management. The problem is deeper than that.
As shown above (from Apple) VisionOS meticulously maps out the environment with LiDAR, cameras, motion sensors, etc. But… why? You can’t align windows to the environment, such as flat against a table, flush against a wall, or aligned orthogonally with your desk. If you want to, you know, put apps in your space, you have to be meticulous. Side-step around for every app, face just the right way, and finely move your fingers until it’s in position. Yet even then it won’t be quite right, like a crooked picture frame.
Unfortunately you can’t align apps with each other, either. Apps launch at some pre-determined distance such that they are facing perpendicular to whichever way you happen to be gazing at that moment. If another app is close enough, it will launch in front of it, with the other one fading out. You can move apps laterally, which causes them to rotate to continue facing the position you happen to be in. You can move apps forwards and backwards, which causes them to proportionately grow/shrink. They don’t actually change in angular size like in the optical illusion below (every image of the car is the same).
With this system, there’s no point in having such flexibility to position apps. It’s basically impossible to line apps up at the same size, uniformly spaced laterally, and at the same depth. Any apps that overlap in terms of line-of-sight (regardless of their position/depth) will cause one to fade out as a portion of one window goes transparent. You have to alternate which one is active/focused by looking at tapping the desired window.
This is all to say that aligning windows is deeply unsatisfying. It’s really the worst of both worlds. It’s not a window-management system designed to be integrated into the environment. But it’s also not designed for an exclusively virtual-positioning system. Instead of just a haphazard collection of randomly sized windows that are just kind of “there”.
Additionally, the scale of app content is just too small. The minimum size of a window (even all the small setting) is probably about the size of a 16” screen, yet shows less content as Than the 12.9” iPad. The novelty of 4’ screens is not so great when they need to be that big to be useful.
Virtual Monitors, Not Windows
David Pierce wrote a great article prior to the Apple Vision Pro unveiling about the potential of the upcoming platform. It can be summed up as if you see a homescreen, they blew it. Well, we got a homescreen.
It’s unweidly having several app windows floating around. They tend to occupy a huge portion of your FOV; 3 windows tends to be a practical limit of working with VisionOS. Turning your head frequently is annoying and prone to repetitive stress. While it’s great to be able to spin up multiple “monitors” whenever you like, it’s silly to need one for every app. Imagine if you had to plug in another monitor for every app you wanted to use on MacOS. It’s not a solution that scales well.
Instead, VisionOS shouldn‘t be launching apps within our space from a homescreen. It should be launching virtual monitors within our space, which can host apps. I’m not talking about MacOS virtual desktops, screen casting, or display mirroring. I’m talking about a simple UI scheme that effectively reserves a plane of space for an app, then lets you stack apps in the same spot. You launch multiple VisionOS apps within the monitor along with a dock to swap between them at the same position. You could have one floating window to work with all your apps if you want. Then at any time you could pull out apps into 2, 3, 4 monitors - whatever you like.
Environments
Environments are the best thing about VisionOS. I love how they are photorealistic rather than 3D generated like a video game. Nevertheless, the execution leaves a lot to be desired.
For one, they are basically apps that launch from the homescreen (also in whichever direction you happen to be gazing at the time). While beautiful and legitimately superior to pass-through video, it’s VR. It means that you suddenly lose all context of your surroundings - meaning you lose the “spatial” in spatial computing. You know all that environ mapping VisionOS is doing? Yeah kinda pointless within an environment. You can only see your app windows floating ambiguously in the nether and your hands (that are poorly masked). As a half-assed compromise, the dial controls the degree of “immersion” so that you can somewhat enjoy an environment while the concept of VisionOS still makes sense.
Instead, environments should open up in a window just like the encounter dinosaurs app. The dinosaur app detects a nearby wall (if available) and opens flush against it, as if the wall no longer exists but is instead a portal to another world. I’d like environments to act similarly - be able to replace a wall with a view into the environment. That way I can work normally at a desk with my virtual monitors, but when I look to the left I see a serene location rather than a view into a parking lot.
Input and Control Center
The eye and hand tracking make for a great combination. I think it was a great choice to go with a solution that did not utilize controllers, but it’s not without its downsides. It does limit the usefulness of VR experiences such as games. Other platforms use controllers to let users to move around a virtual environment (preventing users from bumping into furniture and walls). VisionOS has no such affordance, so any VR experiences are going to have users staying put. Nevertheless, I think they are on the superior side of that equation.
That being said, hand-tracking is really basic and an aspect that makes VisionOS feel oddly rushed. Why is there only a pinch? There’s not really any gestures in the way that iOS has gestures. It’s all just a way-too-literal translation of replicating touchscreen gestures when there is no touch screen. Swiping on a touchscreen on iOS is intuitive a way that pinching-and-flicking in mid-air simply isn’t. I feel like they should have put more thought into gestures that are more natural in context.
Control center is another example of something taken much too literally from iOS. Look up to expose a hidden button that can be tapped? It’s unreliable and its behavior totally inconsistent with open apps. Typically you have to look up with your eyes, but when there’s an app you also need to look well above the app. Yet I’ve also had it appear over videos and panorama photos. Snazzy Labs made the comment about why isn’t something like a a button that appears on your wrist, given the fancy hand-tracking and all. Perplexing complexity
Hampered by Hardware
Although not a review of the Apple Vision Pro, the reality is that VisionOS can only exist through hardware. Right now, it noticeably degrades what VisionOS wants to be. The pass-through video is not high quality, meaning it’s not paritcularly convincing that these virtual apps exist in my reality like magic. Tim says that that Vision Pro “seamlessly blends” virtual and physical object. Well, there are seams, and they are quite noticeable. It’s not so much that it feels like your 2D apps are in the space around you. It’s more that you are keenly aware that you need to be looking through a large contraption to see them.
The eye and hand tracking for VisionOS means that any hardware for it needs to be plastered in all kinds of sensors, cameras, and compute power. Anything that runs it is going to need to be beefy contraption for a long time.
Conclusion
In broad strokes, I think Apple made all the right choices for VisionOS. Positioning it as a platform for a tried-and-true paradigm of 2D apps with controller-less interaction was a great choice. Unfortunately, they fumbled the execution.
More than any other proposal for the future of computing, I’m actually excited by what VisionOS could be. The ability to pull-up virtual, multi-monitors at will on my desk, virtual TV’s on my wall, and virtual windows looking over the ocean.
As-is, however, it’s not all that intriguing - even if I could see it without any device at all. It would be kind of a mediocre experience of siloed windows of huge content, floating just kind of wherever, all getting in each others’ way.
