AI & Virtual Reality—Let The Games (and Work) Begin!

Artificial Intelligence (AI) and Virtual Reality (VR) have enormous potential for entrepreneurs, designers, software experts, video gaming developers and high-tech manufacturers. Some common AI and VR essentials include very powerful computing power in extremely small/tiny spaces and exceptionally high definition displays—real or projected. There will be a requirement for many sensors and actuators too. If today’s marketplace prices are any indication, consumers, businesses and militaries will pay handsomely for the very best. But that presents several challenges including terminology.

AI terminology seems to be described somewhat uniformly but what a given company describes as AI can vary enormously. VR has several common descriptive variants since enhanced reality is more affordable to manufacture and more affordable for consumers. In recent announcements, Microsoft has announced Windows Mixed Reality headsets as part of a new thrust, “We are on a mission to help empower every person and organization on the planet to achieve more, and one of the ways we are doing that is through the power of mixed reality,” said Alex Kipman, Technical Fellow at Microsoft. This is a follow to their earlier HoloLens headsets. The technology behind it will allow its flagship operating system to use the latest generation of Windows 10 hardware devices and software for augmented and virtual reality technologies experiences.

Caption Windows 10 Mixed Reality headsets from partners Lenovo, Acer, Dell & HP


Google’s Alphabet’s X division has moved on to refining their technology with the Google Glass 2.0 headsets the result. Google is targeting business and manufacturing applications that will help boost productivity. The original Google Glass headsets had some appeal but some problems that were addressed for the version 2.0 headsets.

Caption: Google Glass 2.0 headset


Augmented Reality

“AR is a live direct or indirect view of a physical, real-world environment whose elements are “augmented” by computer-generated or extracted real-world sensory input such as sound, video, graphics or GPS data. It is related to a more general concept called computer-mediated reality, in which a view of reality is modified (possibly even diminished rather than augmented) by a computer. Augmented reality enhances one’s current perception of reality, whereas in contrast, virtual reality replaces the real world with a simulated one.[1][2] Augmentation techniques are typically performed in real time and in semantic context with environmental elements, such as overlaying supplemental information like scores over a live video feed of a sporting event,” according to Wikipedia. []

Apple is excited about Augmented Reality and will soon introduce such capabilities soon on iPhones and iPads. There are rumors that they may intro glasses too. What is for real there is Apple’s ARKit, a set of software developer tools for creating augmented reality apps or iOS.

According to Apple developers, “Apps can use Apple’s augmented reality (AR) technology, ARKit, to deliver immersive, engaging experiences that seamlessly blend realistic virtual objects with the real world. In AR apps, the device’s camera is used to present a live, onscreen view of the physical world. Three-dimensional virtual objects are superimposed over this view, creating the illusion that they actually exist. The user can reorient their device to explore the objects from different angles and, if appropriate for the experience, interact with them using gestures and movement.” With their track record on iOS Apps for iPhones and iPads, that may give them an edge.

Not to be overlooked, however, is the now famous and impressive Oculus Rift virtual reality headset, now a Facebook company. It’s still expensive but very competitive. “Content for the Rift is developed using the Oculus PC SDK, a free proprietary SDK available for Microsoft Windows (OSX and Linux support is planned for the future).[62] This is a feature complete SDK which handles for the developer the various aspects of making virtual reality content, such as the optical distortion and advanced rendering techniques”, according to Wikipedia.

With Augmented Reality (AR), one can immerse themselves in games, education, work, movies or whatever. The big question, however, is will this be just a short-term fad or will it be enduring like the personal computer or smartphone? We just don’t know yet but the question of industry-wide standards versus proprietary platforms will likely be an issue. We do know that familiar tech giants are betting on VR and AR for current and future products. What is certain is that thin-film technologies are essential for manufacturing the LCD, OLED or other screens used in the requisite headsets. If you thought that building 4K or 8K HDTVs was challenging, VR and ER in small form factors will make it interesting. The VR headset options abound.

A Crowded Market

“By 2016 there were at least 230 companies developing VR-related products. Facebook has 400 employees focused on VR development; Google, Apple, Amazon, Microsoft, Sony and Samsung all had dedicated AR and VR groups. Dynamic binaural audio was common to most headsets released that year. However, haptic interfaces were not well developed, and most hardware packages incorporated button-operated handsets for touch-based interactivity. Visually, displays were still of a low-enough resolution and frame-rate that images were still identifiable as virtual. On April 5, 2016, HTC shipped its first units of the HTC VIVE SteamVR headset. This marked the first major commercial release of sensor-based tracking, allowing for free movement of users within a defined space. “ per Wikipedia [].

Is AI Ready?

“We are at an inflection point in the development and application of AI technologies,” according to the Partnership on AI []. “The upswing in AI competencies, fueled by data, computation, and advances in algorithms for machine learning, perception, planning, and natural language, promise great value to people and society.

“However, with successes come new concerns and challenges based on the effects of those technologies on people’s lives. These concerns include the safety and trustworthiness of AI technologies, the fairness and transparency of systems, and the intentional as well as inadvertent influences of AI on people and society.

“On another front, while AI promises new capabilities and efficiencies, the advent of these new technologies has raised understandable questions about potential disruptions to the nature and distribution of jobs. While there is broad agreement that AI advances are poised to generate great wealth, it remains uncertain how that wealth will be shared broadly. We do, however, also believe that there will be great opportunities to harness AI methods to solve important societal challenges.

“We designed the Partnership on AI, in part, so that we can invest more attention and effort on harnessing AI to contribute to solutions for some of humanity’s most challenging problems, including making advances in health and wellbeing, transportation, education, and the sciences.”

Some founding members of this Partnership on AI include Apple Inc., Amazon, DeepMind, Facebook, Google (Android, Chrome), IBM (Watson computer), Intel Corp., Microsoft, Sony (PlayStation) and the The Association for the Advancement of Artificial Intelligence (AAAI).

What Is Needed

In future blogs, we’ll revisit some specific AI and VR applications of interest and challenges.