Perhaps it was the moment a virtual butterfly effortlessly landed on my extended finger, or maybe it was the dinosaur's snaggle-toothed maw that came within inches of my face, or even the mountaineer who balanced barefoot on a thin cable pulled taut across a vast ravine. In truth, it was all of those experiences with Apple's stunning Apple Vision Pro spatial computing headset that convinced me I'd just experienced the true future of VR.
I know what you're thinking, "Dude, talk to me when you've washed the Apple Park koolaid out of your system." That's fair. I'm just hours from the moment Apple unveiled its first VR/AR wearable on the WWDC 2023 stage. It was a good presentation but it's hard to convey the power or experience of using VR through a canned presentation; 2D video is not equal to the task.
Think about all those Meta Quest launches with CEO Mark Zuckerberg and his often half-bodied friends wandering around the metaverse. I mean, it sounds cool but... OK, even the experience of the metaverse isn't equal to the idea of the metaverse. Still, to give Meta and other VR purveyors their due, you don't know how good the HTC Vive Pro or Meta Quest Pro are until you try them.
I've tried them all and they all have their moments. They also have their limits. The visuals don't always hold together. Outside-in features like passthrough video are grainy or the representation of your hands and fingers is too cartoony. The headsets are usually too heavy or uncomfortable to wear for more than 15 minutes at a time.
Materials and design
With Apple Vision Pro, a project that was literally years in the making (and it shows), Apple didn't approach these problems by trying to find a midpoint accommodation. Instead, it clearly went all out, sparing no experience on components, design, and material. The result is a mixed-reality headset that is at once familiar and wholly distinct.
It still looks like an expensive pair of ski goggles, but the face is actually an extraordinary piece of glass. In immersive mode and with a kaleidoscope of rainbow colors, the face of it looks like Siri's cousin (yes Siri is integrated, no I didn't get to try it). In passthrough mode, it can show your eyes or maybe a video of your eyes since there's clearly no way to see all the way through the device. They called this feature "Eye Sight." It reminds me of what someone looks like when wearing those old joke X-ray glasses.
The rest is a fabric-covered body and brushed aluminum frame (that joins perfectly with the glass face) and a wide mesh strap on the back to secure it to your head.
Inside is not one but two Apple silicon, the M2 and the new R1 (one handles processing, while the other makes sure the spatial experience is top notch).
You obviously could glean much of this from the presentation and various news reports about the new Vision Pro. I want to tell you about using it.
Apple Vision Pro pricing and availability
Apple announced its Vision Pro headset on June 5, 2023, at WWDC 2023. It will cost $3,499 in the US and ships sometime next year (2024). Availability and price in other markets is yet to be confirmed, but Apple says that will follow in 2025.
Apple Vision Pro setup
Unlike a typical product hands-on where you can do whatever you want with the latest technology, Apple's roughly 30-minute Vision Pro demo was a guided experience. That's not to say I didn't use it. I did, but Apple was quite prescriptive in what I should do. In the end, though, I thought they helped me experience the best of it and maybe gave me my best AR/VR experience ever.
Before I could don the roughly 1lb (453g) headset, though, Apple asked me to get my eyes checked or, rather, my glasses. I walked into a small room where a polite gentleman asked to see my glasses and quizzed me about my eyesight. Was there anything usual about it beyond my progressive lens glasses? I told him there wasn't and he proceeded to stick my eyewear into a system that looked like it belonged in an optometrist's office.
Apple would use this information to select the right set of Zeiss lens inserts for the Vision Pro. Apple told me they had just a fraction of the lens options that would eventually be available to potential Vision Pro customers.
Also in preparation for my first experience, Apple had me use an iPhone to scan my face (in a fashion similar to Face ID registration) and my ears for an accurate spatial audio experience.
With all this done, I waited another 20 minutes before I was led into a room where a pair of Apple representatives would guide me through wearing and using Apple Vision Pro.
Also in the room was a lone Vision Pro. I noticed right away the cable snaking under it to a small but dense battery pack (roughly two hours of battery life) that looked a little like the back of the original iPhone. I also noticed the extra band that ran from one side of the headset to the other that would, it turned out, offer a crucial bit of support when I wore it.
I was not allowed to take photos of myself wearing the Vision Pro headset or capture any of the images I saw (though I promise they would not do it justice.)
Let's get it on
Lifting the headset, I noticed that it seemed smaller and lighter than competing VR headgear from Meta and even HTC. Keeping the battery outside of the body is a less-than-Apple-like move, but I think it's ultimately the right one. A gram or two more in the body and maybe Vision Pro isn't so comfortable to wear.
I was instructed to grab the Vision Pro by the area that would sit on the bridge of my nose and by the wide mesh on the back. I placed it on my face and it started to slide down my nose. There's a wide ring near the back that I then used to tighten Vision Pro on my face. I was cautioned not to make it tight. I twisted the dial until the headset felt firm and, more importantly, balanced on my nose and forehead. Then I tightened the strap that ran over the crown of my head. That was what did the trick. Now the Vision Pro felt snug and comfortable.
This is where things got interesting.
The system launched with the classic Mac "hello" drawing out in 3D script in front of me. It's a nice touch that neatly ties this new hardware category to Apple's iconic product history.
Apple Vision Pro is mostly controlled via gesture and vision tracking but, in order for that to work, I had to go through a brief setup routine. For the eye tracking, I was instructed to keep my head still and then look at a series of white dots that floated one at a time before me. I did this twice. Next, I held up my hands in front of my face. The system's multitude of cameras registered them in seconds. As it did so, I noticed a faint, shimmering glow around them.
I can see clearly now
Did I mention that the Vision Pro has excellent passthrough vision? Instead of a grainy or limited view of the outside world, it looked almost as if I was peering through clear glass. Throughout my demo, this view would purposely phase in and out of visual existence. Apple intentionally does what it can to keep the real world and people around you visible, unless you are in fully immersive mode.
There are a pair of buttons on the Vision Pro. The one on the upper left is for capturing spatial (3D) photos and videos. I never used that. On the other side is the Digital Crown. Yes, it's just like the Digital Crown on an Apple Watch but larger. When I heard Apple included this on the Vision Pro, I was skeptical. Why would Apple bring a tiny Apple Watch part to its VR headset? Turns out, it's a near-perfect way to augment gesture controls.
I pressed the Digital Crown to reveal the headset's main interface (you can also press and hold it to recenter the main interface). As I scanned the small set of horizontally aligned apps that looked like a cross between iPad and iOS icons, I noticed that each one kept almost pulsing forward toward me. It took me a beat to realize that they were each reacting to my gaze.
One of the Apple reps explained how I would control Vision Pro through gaze, combined with pinches and horizontal and vertical pulls. Basically, if I looked at an app like Photos, I could then pinch together my thumb and index finger to open it. To scroll in a window, I would pinch, hold and drag my hand left or right or up or down. There was never a need to carefully hold my hand in front of my face. Usually, I was resting my right hand in my lap when I made a pinching gesture.
I eventually used these gestures to open and close windows, select photos, and scroll through open Safari pages. With little guidance, I quickly moved various app windows (Messages – I sent a text message without any guidance, Photos, Safari) all around the room to create the biggest desktop I'd ever seen.
I remember years ago my first experience using Microsoft Hololens. I liked it. It was a big leap forward in the world of mixed reality, but there was also no getting past its limits like a narrow field of view and sometimes ineffective gesture controls.
Apple Vision Pro's field of view is, by contrast, as big as the space around you. Interface screens and massive windows appeared to fill the room.
In the Photos app, I looked at 12ft photos and panoramas that basically wrapped almost fully around me. Apple, I thought, had finally found a use for all those panoramic photos we've been taking for almost a decade.
One of the odder experiences was a FaceTime call with an Apple spokesperson. The call alert appeared way at the top of my virtual screen and almost out of view. I looked up at it and pinched to open the call where I was greeted by an eerie-looking "Persona" avatar. This is the 3D scan of a face that you can capture with the Vision Pro and then use in your own Vision Pro-based FaceTime calls. As the Apple rep explained, the avatar represents where she looked, and her expression, and lip-synced her voice with the avatar's animated mouth. It looked not quite human and was my least favorite part of the experience.
On the bright side, she showed me how we could collaborate through FaceTime on a Freeform project board. It was an impressive bit of collaboration in 3D space.
Remarkable displays
Imagery in general is fantastic thanks to the dual micro-LED displays that provide, according to Apple, 23 million pixels of imagery. What I noticed is that no matter where I looked or even if I glanced at the edge of the images, they never distorted or faded. I did see some light leaks, especially in dark images but never when I was fully immersed in some experience. Apple said they expect to have more custom light seal options when Vision Pro ships.
All the photography, much of which was taken with an iPhone, looked great, but we slipped just a tad into an uncanny valley with Spatial Photos and video. I'm not saying these images and videos of a child blowing out her candles (with smoke virtually hitting my face) and friends gathered around a fire pit didn't look real. They looked more than real. It was alike a postcard from Minority Report but instead of pre-cogs, we have a past-cog Vision Pro letting us relive moments like never before.
Things only got wilder from there.
Apple showed me (or guided me to open) Environments, which are like 360º photos or backgrounds. I could use the Digital Crown to dial up or back the level of immersion. It was at this moment that I happened to look down at my hands. They were resting on my knees which had, well, disappeared. Instead, my real hands and forearms were resting on the immersive background. It was a startling effect.
Entertain me
Vision Pro will clearly have many uses. It has the potential to be the ultimate productivity environment, allowing you to use a physical keyboard and mouse with a massive virtual desktop (I did not get to try any physical devices with it), and has obvious entertainment potential.
Apple directed me to launch a 30-second 3D clip of Avatar 2, which loaded up on a large screen that floated in our meeting space. The movie looked beautiful, vibrant, and enchanting. The 3D effect was quite good. Then we selected a Cinema Environment and the room faded to black and it looked as if Avatar 2 was playing on a giant movie house screen. This significantly enhanced the overall 3D effect.
There is, however, a big difference between watching a movie and, essentially being inside of one.
Apple's Immersive Video is Apple's own spin on immersive VR experiences. A series of short clips put me feet away from Alicia Keyes singing a set in a small studio, in flight over mountains and skyscrapers, and apparently standing on a thin cable over a ravine. On that last bit, I wasn't alone. Right in front of me with her arms outstretched stood a barefoot woman carefully balanced on the cable. It was thrilling and terrifying.
This is different
My final Vision Pro experience, though, was unforgettable. I opened an app called Encounter Dinosaurs and then watched as the wall in front of me parted to reveal a barren and ancient landscape. I spotted a small butterfly on one of the rock outcroppings. The Apple rep told me to hold out my finger. I did as I was told and the butterfly alighted, fluttered, and then flew to me, carefully landing on my outstretched digit. I gasped.
The butterfly flew off and then a small dinosaur crawled out of a crevice. The spatial audio, which had been so good and so subtly on point throughout my demo experience, made it obvious that something large was approaching from the left, just behind the office wall. A large reddish-brown dinosaur thunderously emerged. It glanced about and then looked directly at me. Soon it was walking toward me and I reflexively pressed my back into the couch. The Apple reps encouraged me to get up and approach the dinosaur. Of course, I had to pick up the small battery back.
I stood and, since I could still see the whole room around me, walked around the coffee table. The dinosaur watched me, warily tracking my movement. Soon my face was just inches from its toothy mouth and snout. It was amazing, a dinosaur lover's dream. Then the dinosaur retreated back to the rocky surface and then the wall closed as the dinosaur let out one last roar.
Early verdict
Apple has built the first lust-worthy VR headset. It's beautiful to look at and gets most of the key VR and AR experiences right. Even now, months from release, it's already the most intuitive VR interface not yet on the market. The eye and hand tracking are already excellent. I think spatial photos and, especially video, may change how we engage in memories,
There are some issues and hurdles.
The EyeSight display that can show you someone's virtual eyes is probably a mistake, as may be the weird 3D Persona avatars. It's also, at $3,499, wildly expensive, and could be more so if you need to buy Zeiss lens inserts (maybe one set for each family member). Apple may struggle to justify that expense unless it can get a whole lot more people to experience it. It's quite possible that by this time next year, there will be dozens of Apple Vision Pro headsets available at Apple Stores around the world for hands-on experiences.
That may change a lot of minds and turn the Apple Vision Pro into the ultimate wishlist VR product. Can it catch up to the popular Meta Quest or HTC Vive? That all depends on how willing people are to part with thousands of dollars for what is clearly an extraordinary, premium-level VR experience.
