Demystifying the AR and VR technology

Augmented Reality (AR) and Virtual Reality (VR) are emerging technologies with huge potential to change the way we interact using information, visualization and interaction. This is applicable for both consumer and enterprise use cases. Mixed Reality (MR) is a combination of AR and VR, merges images with existing real-world objects to augment or enhance a user’s experience. Add to this confusion, another terminology Extended Reality (XR). widely used which refer to all types of reality – AR, VR and MR.

Internet of Things (IoT) is an intersection of Physical and Digital, hence it is also referred to as Cyber-Physical System. When complimented with XR, IoT can result in more innovative, highly immersive experience.

This is where everything gets blurred, not able to connect the technologies to the application and use cases. In this writeup I have tried to interpret the different XR technologies with examples and how IoT can complement and augment the experiences. This writeup does not cover in depth the specific technology challenges nor the enterprise use cases.

Exploring the spectrum of reality

On one end of the spectrum there is reality that is closer to the physical world, to the extreme end of this is the virtual world which is complete simulated away from reality. For a long time Virtual Reality was existing in the form of games, flight simulators, and so on. The diagram below depicts the Reality spectrum from Real to Virtual.

Reality Spectrum

AR technology is about super-imposing the data from real world onto a real world objects, for eg., eyewear application for trialing different glasses. VR technology is about illusion and synthesized information and environment, example gaming.

As the technology matured especially powerful processors, GPUs, display and more importantly head mount display and the explosion of mobile virtual reality experience became more interactive and immersive.

To understand this better, the following table will probably clarify the different Reality technology and their differences, its scope.

Type of Reality Use case examples
Augmented Reality Used to augment real world with rich data and insights.

  • Hawk-eye technology that visually track the trajectory of the cricket ball, used in DRS and also bowling analysis.
  • Superimposing of the lane information on the windscreen
  • Pokémon GO gaming app where characters are found in real world
  • Skygazing apps that place the planets and the stars along with the constellations in the real sky
Virtual Reality Predominantly used in gaming and training. Target base is mostly consumer and entertainment.

  • Games like Temple Run, Angry Birds, etc.
  • Normal Flight Simulators we play in PC or mobile phones
Augmented Virtuality Training and entertainment is the primary application.

  • The same flight simulator with a motorized chamber with 6 degree of freedom movement and haptics, with landing in a real airport strip
  • Universal Studios Harry Potter Ride
  • Training in the Medicare, for eg., a force feedback laparoscopic training device.
  • Snapchat phot app editor and filter to create funny photos and videos

Augmenting XR with IoT

With Internet of Things (IoT) the lines are blurring between digital and physical. With the advancement in sensor and connectivity technology along with cost reduction, it is feasible and viable to engineer smart connected product. What this will enable is to get the real-time data from the physical systems augmenting and amplifying the XR immersive experience. The diagram below captures the intersection of IoT with Real and Virtual ecosystem.

Intersection of IoT and AR/VR Technology

The table below explains how IoT technology can augment and amplify the AR-VR experience.

Mode of Reality Enhanced by IoT Use cases
Augmented Reality
  • Real-time data from the physical asset
  • Closed loop feedback to the real physical asset for command and control
  • Sensing of movement in real world using sensors (GPS for eg.) and vision based localization
  • Remote operations of Machine, Plant and Factory Operations.
  • Remote diagnosis of the machines and equipment.
  • Equipment and machine service management
  • Digital Twin of any physical system. For eg., it can be machine like turbine, car, truck, home appliance, building, etc.,
Virtual Reality
  • Haptics enabled joystick, mouse, or even head mounted display
  • Hand, Head and body movement tracking using sensors.
  • Use of Spatial Audio integrated with body and head movement
  • Gesture Tracking through devices
  • Enhanced gaming devices for feedback. For eg., PS4, Xbox, Wii
  • Virtual tour of building, venues, exhibitions, etc.
  • Immersive movie experience
Augmented Virtuality
  • Hand, Head and body movement tracking. with accelerometer and gyroscope.  Representing movement into the virtual space to give immersive experience
  • Gesture Tracking through devices
  • Robotics for multi-degree freedom movement.
  • Simulated feedback in training through and movement similar to real world
  • For eg., Nintendo Wii, Flight Simulator, 3D movies

The three senses that are leveraged for rendering an immersive XR experience are Vision, Audio and Touch.

  • Vision – The visual rendering can be done using Mobile, Desktop, Projector Display, Video See through Display, head mounted display (also referred as Head-Up Display, HUD) and holographic rendering. Some of the popular head mounted display are Oculus, HTC Vive, Realware, and so on. The head mounted display restricts the free movement and also causes cybersickness due to occlusion and prolonged viewing
  • Audio – Similar to head mounted display, airpod/earbuds are used for rendering spatial audio sometime in 3D space. Head and body movement monitored using accelerometer and gyroscope is used to render the contextualized spatial audio. Dolby Atmos is one such technology standard. Apple Spatial Audio, Sony 360, Sony PS5 with pulse 3D wireless headset are some of the products.
  • Touch – Simulated feel is rendered using haptic technology and devices. Closed loop mechanism provides feedback using actuators feeding vibration and synthetic touch. Phantom Omni from Delft Haptics lab and TouchX from 3DSystems are some of the popular Haptic devices.

Summary

What is the status of the various XR technology and what does the industry trend shows ? Gartner (Source:  https://twitter.com/SkarredGhost/status/1396549995621273603/photo/1)  Hype Cycle has been tracking the maturity of the AR and VR technology over the years. This diagram below captures only the AR/VR technology in the cycle. The VR has seen in the “slope of enlightenment”, however the AR is still in the “Trough of disillusionment”.

 

Gartner AR-VR Hype Cycle

There are multiple market research and statistics about the growth of AR/VR handset and devices. According to a report from tech market research firm IDC (https://www.idc.com/promo/arvr), AR and VR headset shipments will reach 9.7 million units by the end of 2021 and are expected to grow to 32.8 million units by 2025 with a compound annual growth rate (CAGR) of 45.9%. These are customized AR/VR devices and excludes the mobile phones, tablets that already render AR/VR.

Finally to summarize, there is a lot of scope of scaling up of use cases especially in the enterprise. In the manufacturing domain AR can improve the operations, training, service management. While there is lot of advancement in the IoT to get the data from the machines, the AR technology around display has to improve. VR has seen more adoption in enterprises for training, simulation, design, construction and many more. With advancement in display, resulting in reduced cybersickness and improved mobility, Richer data with IoT, especially spatial and temporal data, closed loop through actuator control,  AR/VR will see large scale growth in enterprises.

 

Author Details

Krishnananda R. Shenoy

Krishnananda R Shenoy is the Chief Architect of Internet of Things (IoT) practice, Engineering Services at Infosys. Responsible for solving customer IoT requirements and solve the problems by proposing the right technology solution. This includes Edge, Cloud and Hybrid architecture. Industry 4.0 Consulting and Advisory for helping customers transform towards digital manufacturing. Implement emerging technologies like AR/VR, Digital Twin, 5G and Industry Cyber Security. Responsible for setting up Go-To-Market with product partners, hyperscalers and startups.

Leave a Comment

Your email address will not be published. Required fields are marked *