Expert Perspectives #1: How apple vision pro is transforming digital experiences

Welcome to our Expert Perspectives series

This series features in-depth interviews, offering insights into cutting-edge technologies and their impact on various fields. In our first edition, we explore the transformative potential of Apple Vision Pro and discuss how this device is set to revolutionize digital experiences through its advanced augmented reality capabilities. For this interview, we spoke with Patrick Gaißert, a recognized expert in the field, who shared his extensive knowledge and experience to help us understand the innovations, applications, and future implications of Apple Vision Pro. Now, let’s hear directly from Patrick, on the capabilities of Vision Pro, starting with his perspective on how it compares to other AR/VR devices on the market.

Interview with Patrick Gaißert:

How does Apple Vision Pro differ from other AR/VR devices on the market?

“In my opinion there’s three main points which set Apple Vision Pro apart from other AR/VR devices on the market:

• First is the seamless and visually stable integration of digital content into the physical world. This is enabled by its very high quality displays as well as Apple’s attention to detail when it comes to operating system design. Virtual windows cast shadows and 3D objects reflect the real environment around the user.

• As a second point it successfully shapes a user interaction paradigm based on eye tracking and simple hand gestures, which is effective and efficient.

• Third being the compatibility with iPhone and iPad apps running directly on the device. This allows for existing workflows to transfer over to the new platform immediately. In the future this also works the other way around: a Vision Pro app can also support iPad for example, in order to make the solution available to more users.”

How will Apple Vision Pro change the way people interact with digital content/products?

“The first time someone tries out Vision Pro is transformative in my experience. You have to try it out for yourself to truly get it. It changes your expectations of what it means to experience digital products by embedding them in the user’s physical environment. The device’s eye tracking and gesture controls enable rapid, mostly natural interactions which allow you to do virtual try-ons of shoes or watches for example. By being the first XR platform to really focus on daily-use applications, Apple Vision aims to become an integral part of everyday activities, both at home and in the workplace. While the device’s weight and dimensions still need to be reduced over the coming years, the first model is already proof that immersive technologies like it are able to create emotional connections with digital content that cannot be achieved with traditional two-dimensional screens.”

What new interaction methods are possible with the Apple Vision Pro that were previously unimaginable?

“Users control focus through eye movements, enabling interactions as fast as the eye can see. Hand gestures are the primary mode of input, replacing traditional hardware controllers. The operating system’s ability to anchor and remember where you put digital content within your apartment, for example, allows you to create contextually relevant spaces. If you place a cooking app in your kitchen and your favorite movie app in the living room, they will still be there the next time you enter these rooms.

Moreover I like the idea of what I like to call “bi-directional presence”. Displaying the user’s eyes on the headset as well as people in your vicinity always shining through any virtual content is supposed to reduce the potential isolation of wearing a device like it. While this feature is in its early stages and pretty rough at the moment I still think the subject-matter is important since, for example, it allows you to reasonably interact with your coworkers while wearing it.”

How can technologies like Apple Vision Pro contribute to making digital design more human-centered and intuitive?

“From an evolutionary perspective it makes total sense to me to base a spatial computing platform around natural human behaviors like looking, gesturing and speaking. We learn these interactions from a very young age and use them all day every day. While touchscreens turned out to be really intuitive for most users I think this new paradigm has the potential to be even more powerful.

When I first started using the device I was surprised that most of Apple’s system apps are purely window-based. I realized over time that this is not laziness on Apple’s part but just good design. The platform fully supports window-based applications in addition to fully immersive use cases and anything in between. This allows digital designers to design applications purely based on suitability for the task (ISO 9241), not every use case profits from being immersive! Another important aspect is accessibility, which is sadly often treated as an afterthought especially in XR. Vision Pro on the other hand has the most comprehensive accessibility support of any Apple platform at launch. I think this is one point where Apple really doesn’t get enough credit.”

Can you think of surprising yet valuable examples of how the Apple Vision Pro might be utilized across different industries?

“One thing to keep in mind is that the first model is only designed for indoor use. That being said there are already trials in the healthcare industry, where it could enhance surgical procedures by overlaying critical information onto the surgeon’s field of view. In industrial manufacturing, it could provide workers with real-time data and instructions, improving efficiency and safety. In education, it could bring immersive learning experiences to students, allowing them to explore historical events, scientific concepts, and art as if they were physically present. In retail, customers could try on clothes virtually or see how furniture looks in their home before making a purchase. With upcoming models it could also extend to outdoor use cases, for example in the field of architecture and construction, professionals could use the device to visualize buildings and structures on-site before they are built, facilitating better planning and collaboration.”

What potential challenges do you see in implementing this technology?

“In my opinion the main challenge lies in addressing ergonomics to make the device comfortable for prolonged use. There’s no way around it: the first model is heavy and bulky especially for people with a smaller head. In the consumer space, the main challenge will be communicating the value of the platform to customers. You really have to experience it to believe it. The high cost will initially stand in the way of widespread adoption, although various model variants of the Apple Vision platform are very likely in the coming years.”

What ethical or inclusion challenges do you see with technologies like Apple Vision Pro?

“Like I already mentioned, the accessibility support on Vision Pro is outstanding in terms of features. It offers comprehensive VoiceOver screen reader support for both window-based and immersive experiences, including apps built using the Unity engine. While the user experience will vary depending on the specific app’s implementation, visually impaired and blind users have full access to the device and its apps. Additionally, Vision Pro supports a range of known Apple platform features, such as text size adjustment, zoom, switch/voice control, sound recognition, and guided access, among others. Platform-specific options include Dwell Control, which enables device usage with just one or two eyes, without the need for hands. By focusing on a UI element for a few seconds, users can activate it. An upcoming feature is Live Captions, which displays subtitles while conversing with others, making it possible for deaf or hard of hearing individuals to follow along.

However, there are currently some accessibility issues especially during the initial hardware setup. For example, for users having physical difficulty pressing the Digital Crown on top of the device. These issues need to be resolved through software updates. Personally, I believe the Apple Vision platform holds great potential, especially for users with visual impairments. While the first version is not designed for on-the-go use, this may change with future models. The device’s advanced scene understanding capabilities could provide auditory hints about the user’s environment, such as precise location and labeling of doors, signs, stairs, and hazards. This could complement or potentially even replace a blind person’s cane.

From an ethical standpoint, these capabilities could lead to an overdependence on assistive technology which, definitely for Apple products, is not affordable for everyone. However, in my opinion this issue is not limited to this platform and is a broader, more general concern.”

How do you see AR/VR technologies developing over the next five to ten years?

“In the next ten years, I expect Spatial technologies to evolve significantly, with devices becoming progressively more lightweight and compact. Performance improvements will further enhance the integration of Artificial Intelligence features as well as field of view and sensor accuracy, leading to even more realistic and engaging experiences. As the technology matures, we can anticipate broader enterprise and consumer adoption, with Spatial Computing becoming integral to various aspects of daily life, including outdoor activities, navigation and fitness.”

What are your visions for the future of human-machine interaction?

“The future of human-machine interaction will likely be characterized by more natural and intuitive interfaces that blend seamlessly into our daily lives. Advances in spatial computing will enable us to interact with digital content as part of our physical environment, making technology an extension of our natural behaviors. As devices become more aware of our context and surroundings, they will anticipate our needs and provide assistance in increasingly subtle and ambient ways. This evolution will lead to a future where the boundaries between the digital and physical worlds become increasingly blurred.”

Summary: Embracing the Digital Future

We hope that these insights have opened new perspectives on the future of digital interaction. Thank you to Patrick Gaißert for sharing his expertise and vision. We look forward to seeing how AR/VR technologies evolve and what new experiences they will bring us in the years to come.