Final version of Vision Pro, Overlay or Pass-through

[ZombiePhysicist]

I think for mass adoption the AVP needs to become RayBan sized glasses. My question is, do you think the final version will have an augmented overlay, where you see the real world through a transparent lens (a bit like some of the hololens's designs) or will it be a full video feed recreation like in the current AVP.

So these are RayBan style glasses:

These are goggle style:

AVP currently employs video passthrough (your yes look at screens) where as some systems like HoloLens or the original google glass let you see reality through a clear lens and overlay graphics onto the glass, but you still see the real world directly, not via video feed:

One of the benefits with passthrough video is you can more easily augment "reality" as it is already a feed, versus somehow overlaying what you see seamlessly with a transparent piece of glass.

Considering some of the brain altering effects of pass through video (not to mention nefarious subliminal effects that can be programmed into such a device to reprogram your brain over time), one might think they will eventually move to an overlay.

So why didnt they do that with this product? My guess is because the tech was just not there to do all they wanted to do. It would be far more difficult to map a window back in 3D space with an overlay, I suspect.

Curious what others think so I set up poll!

 

[T'hain Esh Kelch]

What is the difference between the Rayban and Google versions? Just a cool design and one not?

 

[ZombiePhysicist]

What is the difference between the Rayban and Google versions? Just a cool design and one not?

I updated the original post to give some examples.

 

[AlastorKatriona]

These are 2 completely different products. Vision Pro is a VR headset with fake passthrough. That's not even in the same conversation as a pair of AR glasses that are augmenting the world.

 

[ZombiePhysicist]

These are 2 completely different products. Vision Pro is a VR headset with fake passthrough. That's not even in the same conversation as a pair of AR glasses that are augmenting the world.

Certainly not an unreasonable take.

That said, passthrough very much also does augment the world. It uses different tools to do the same thing. I would argue a pure VR device has no passthrough and you are just in a virtual space all the time. Maybe that's too old school a take and the definitions are evolving.

That said, that one can be a starting point for another, IMO, is also not an unreasonable take.

 

[hippo206]

I could see both. Glasses for quick everyday use, goggles for deeper immersion.

 

[T'hain Esh Kelch]

I updated the original post to give some examples.

Aaah… Goggles, not Googles version of these…

 

[gerald.d]

There is no "one size fits all" product on the horizon in the foreseeable future, unless someone makes significant progress in beaming an image directly onto the retina with lasers (there are actually products out there already doing this, such as the Sony HX99 RNV camera kit).

There will always be two completely separate devices because there are two completely separate use cases in the larger VR/AR/XR/Spatial Computing/or whatever you want to call it space, and until full retina projection is feasible, the two use cases cannot be delivered with a single device.

Fully immersive goggles for use at home and in some work environments (or sat on long journey where you're not the one controlling the vehicle) with video passthrough, and glasses for use when out and about with projected screens overlaid on the real world.

 

[AlastorKatriona]

Certainly not an unreasonable take.

That said, passthrough very much also does augment the world. It uses different tools to do the same thing. I would argue a pure VR device has no passthrough and you are just in a virtual space all the time. Maybe that's too old school a take and the definitions are evolving.

That said, that one can be a starting point for another, IMO, is also not an unreasonable take.

Right, but what I'm saying to you, is that fake passthrough created by cameras and screens....and real passthrough, looking through a lens, are so far apart from each other that they will never been interchangeable. They enable completely different products. Cameras + screens will NEVER be a replacement for looking through a clear lens. Never. Not in our lifetime.

 

[dwhite601]

These are two different products with two different missions. Neither can cover all the use cases well.

The problem with headsets is weight and comfort so that would imply that glasses would be better once we get the required resolutions. The problem with glasses, though is occlusion. In mixing graphics pixels with optical light sources, the brightest pixel wins. Imagine trying to watch a video while at the beach mid day. Dark scenes in the video will be lost in the bright environment. A cover could be put over the glasses, but light will still leak in from the sides. Cover those and your right back to wearing a headset.

On the other hand, simple glasses could easily display bright overlays of real world objects that would be useful for some applications.

What I like about the headset approach with its pass through cameras is graphical objects can appear in front of, as well as behind of, real world objects making them far more realistic. This can't be done with an optical pass through.

David

 

[CrysisDeu]

Goggle with pass through will never be successful

 

[Dovahkiing]

Right, but what I'm saying to you, is that fake passthrough created by cameras and screens....and real passthrough, looking through a lens, are so far apart from each other that they will never been interchangeable. They enable completely different products. Cameras + screens will NEVER be a replacement for looking through a clear lens. Never. Not in our lifetime.

I agree with this 100%. What’s weird about AVP though is it’s like Apple can’t decide which they’re trying to be. To me, it seems like a primarily an AR product with cameras and displays trying its best to do what AR glasses would excel at. And because of that, I do not think the poll is comparing apples and oranges - I very much believe future iteration of AVP will be AR glasses not displays/cameras. As you implied, camera/displays will not replicate glass in our lifetimes I think.

 

[CrysisDeu]

I agree with this 100%. What’s weird about AVP though is it’s like Apple can’t decide which they’re trying to be. To me, it seems like a primarily an AR product with cameras and displays trying its best to do what AR glasses would excel at. And because of that, I do not think the poll is comparing apples and oranges - I very much believe future iteration of AVP will be AR glasses not displays/cameras. As you implied, camera/displays will not replicate glass in our lifetimes I think.

Yeah, and that’s where Apple made the wrong call. They cannot slowly evolve AVP into a pair of glass, that’s not gonna happen. The difference is too large they are not even the same product category. All of the researches, Os, framework, apps, will not work for a pair of glass.

They spent so much effort chasing pass through, but pass through will not be good enough for at least a decade. And so much of the cost and the weight is wasted on the need for achieving pass through

 

[ZombiePhysicist]

Yeah, and that’s where Apple made the wrong call. They cannot slowly evolve AVP into a pair of glass, that’s not gonna happen. The difference is too large they are not even the same product category. All of the researches, Os, framework, apps, will not work for a pair of glass.

They spent so much effort chasing pass through, but pass through will not be good enough for at least a decade. And so much of the cost and the weight is wasted on the need for achieving pass through

I'm not sure I agree with the above sentiments. I suspect there will be some overlap in applicability. Maybe you guys could share where/why what tech areas make the rift as insurmountable as you suggest.

As for pass through video not being good enough for a decade, I definitely disagree. Right now it's at about 20/30 level vision with the 23MP worth of display. This seems like a move from roughly 4k to 8k screens should get you to about 20/15 vision, which really should be pretty good. This seems in the realm of feasible within 3-4 years. But I may be off in my estimations.

 

[Jensend]

Final version of Vision Pro
What do you think the 'masses' form factor will be for Apple Vision Pro eventually?

Apple may make a different style of face mounted computer. But it won't be called a Vision Pro.

 

[CrysisDeu]

I'm not sure I agree with the above sentiments. I suspect there will be some overlap in applicability. Maybe you guys could share where/why what tech areas make the rift as insurmountable as you suggest.

As for pass through video not being good enough for a decade, I definitely disagree. Right now it's at about 20/30 level vision with the 23MP worth of display. This seems like a move from roughly 4k to 8k screens should get you to about 20/15 vision, which really should be pretty good. This seems in the realm of feasible within 3-4 years. But I may be off in my estimations.

I think it’s regardless of the camera and the screen. The fact that your eyes can focus only on one plane all of the time is a problem.

 

[AlastorKatriona]

I agree with this 100%. What’s weird about AVP though is it’s like Apple can’t decide which they’re trying to be. To me, it seems like a primarily an AR product with cameras and displays trying its best to do what AR glasses would excel at. And because of that, I do not think the poll is comparing apples and oranges - I very much believe future iteration of AVP will be AR glasses not displays/cameras. As you implied, camera/displays will not replicate glass in our lifetimes I think.

When you understand that Apple set out to make a real AR product, and failed because the hardware isn't possible (yet), and settled on this, it makes more sense. This is more like a demo or proof of concept of what they set out to make.

 

[ZombiePhysicist]

I think it’s regardless of the camera and the screen. The fact that your eyes can focus only on one plane all of the time is a problem.

How do you mean? The AVP defiantly tricks youre eyes to focusing to different planes despite that the physical plane is singular. It tracks your eyes and refocuses where you look wrt to video being refocused. It's one of the things put in my observations post, that it essentially gives you (crappy) bionic vision.

 

[CrysisDeu]

How do you mean? The AVP defiantly tricks youre eyes to focusing to different planes despite that the physical plane is singular. It tracks your eyes and refocuses where you look wrt to video being refocused. It's one of the things put in my observations post, that it essentially gives you (crappy) bionic vision.

So yeah the camera or the algorithm refocuses the passthrough video, but your eyes don’t refocus, as the screen is at a same distance. that is inherently different from the real world

 

[subjonas]

How do you mean? The AVP defiantly tricks youre eyes to focusing to different planes despite that the physical plane is singular. It tracks your eyes and refocuses where you look wrt to video being refocused. It's one of the things put in my observations post, that it essentially gives you (crappy) bionic vision.

So yeah the camera or the algorithm refocuses the passthrough video, but your eyes don’t refocus, as the screen is at a same distance. that is inherently different from the real world

I had some confusion about this and as I understand it now, with headsets the lens between your eye and the screen causes you to focus past the screen at a fixed plane about 6 ft in front of you, depending on the headset. So your eyes are always focused at that distance regardless of where virtual objects appear to be. Not sure how that part works though.

 

[CrysisDeu]

I had some confusion about this and as I understand it now, with headsets the lens between your eye and the screen causes you to focus past the screen at a fixed plane about 6 ft in front of you, depending on the headset. So your eyes are always focused at that distance regardless of where virtual objects appear to be. Not sure how that part works though.

Yes, that was an issue for VR for quite a while and may cause eyes to lose focus. I know some headsets can dynamically adjust focus to some extent but I don’t think AVP is doing this.

My point is that is we go through the video pass through route, there are so many things to handle. It’s simply physically not possible to solve it in a device that one would like to wear every day in the format of a headset

 

[CrysisDeu]

When you understand that Apple set out to make a real AR product, and failed because the hardware isn't possible (yet), and settled on this, it makes more sense. This is more like a demo or proof of concept of what they set out to make.

Apple is not the company to make, release and sell proof of concept things

 

[CrysisDeu]

I'm not sure I agree with the above sentiments. I suspect there will be some overlap in applicability. Maybe you guys could share where/why what tech areas make the rift as insurmountable as you suggest.

Oh and sorry I missed the first part of the question. Let me explain.

AVP had a few hardwares which the sole purpose is to support pass through. With that, a suit of software and software features to support those hardwares. These, are not really useful for a pair of glass.

A pair of glasses, on the other hand, have other areas of challenges like dimming the background, visibility, etc.

On the use case side, the main use case for a goggle is inside, half or fully immersed. The main use case for a glass is daily use/outside. The applications, requirements, interaction models are distinctive different.

And small differences on two distinct product categories can be enormous technical challenge.

 

[dwhite601]

I think some people are using the term focus when they mean converge. Convergence is rotating one's eyes towards each other, for example, to view a close object. It's simple to mix focus and convergence up since, in normal vision, we always focus and converge to the same distance.

3D video, on the other hand, requires us to converge at different distances while staying focused at fixed distance. This is the same for any 3D display technology.

My theory is that having to learn to focus and converge at different distances at once is what contributes to 3D video head aches. I've found that I've gotten used to it and 3D gets less stressful with experience.

David

 

[Dovahkiing]

Agree with a lot of the above - this is why I think cameras and pass through is not the future.