Global AV & Advanced Driver Assistance Efforts Gain Momentum

Fully AVs (Autonomous Vehicles, a.k.a., Driverless Vehicles and ADS/ADAS advanced driver assistance systems) are still works in progress with different global proponents pushing competing technologies and strategies. Technical hurdles certainly remain as do legislative agendas to control the AVs or near-AV ADS vehicles already roaming some of the streets worldwide in tests.

Until AVs are around in large numbers, however, we will likely see more assisted driving ADS vehicles that require the driver to pay attention and intervene as needed per the vehicle’s rules. For ADS, some carmakers are looking at implementing head-up displays (HUDs) on the windshields, much like some military fighter and commercial aircraft pilots have had for some time. This permits critical information such as speed or warnings to be displayed without drivers taking their eyes off the roads and nearby traffic. Unlike military or commercial aircraft applications, HUD costs will certainly be major considerations in vehicles. Of course HUDs could evolve much like TVs where the old grainy pictures of the past have evolved into affordable HDTV and 4K HDTVs—incredible resolution at prices that keep dropping.

Caption: The NXP S32 Auto Processing Platform

 

NXP Semiconductors N.V., one of the world’s largest suppliers of automotive semiconductors, has a control and computer system for connected, electric and autonomous cars. The NXP S32 platform claims to be the world’s first fully-scalable automotive computing architecture. Soon to be adopted by some premium and volume automotive brands, it offers a unified architecture of microcontrollers/microprocessors (MCUs/MPUs) and an identical software environment across application platforms.

This system addresses the challenges of future car development with an architecture that allows carmakers to provide custom in-vehicle experiences while bringing automated driving functions to market much faster than before.

“Traditional and disruptive automakers, even more than Tier ones, seek a standardized way of working across vehicle domains, segments and regions to meet increasing performance demands while contemporarily ensuring fast time to market and control over skyrocketing development costs,” said Luca DeAmbroggi, senior principal analyst, Automotive Electronics & Semiconductors at IHS Markit in London. “A common architecture and a scalable approach can cut development time for critical applications in domains like ADAS, autonomous driving or connectivity from both the HW and the SW perspective.”

Vehicle Automation—AI Enabled

AI itself is getting much smarter as a recent MIT Technology Review news item explained. “AlphaGo Zero Shows Machines Can Become Superhuman Without Any Help. It explains that AlphaGo wasn’t the best Go player on the planet for very long. A new version of the masterful AI program has emerged, and it’s a monster. In a head-to-head matchup, AlphaGo Zero defeated the original program by 100 games to none.

“What’s really cool is how AlphaGo Zero did it. Whereas the original AlphaGo learned by ingesting data from hundreds of thousands of games played by human experts, AlphaGo Zero, also developed by the Alphabet subsidiary DeepMind, started with nothing but a blank board and the rules of the game. It learned simply by playing millions of games against itself, using what it learned in each game to improve.

“The new program represents a step forward in the quest to build machines that are truly intelligent. That’s because machines will need to figure out solutions to difficult problems even when there isn’t a large amount of training data to learn from,” per the Technology Review description.

Machines that are aware and learn are not new but this iteration is very impressive. Since AI and Machine Learning (ML) are essential for advanced AVs, this may accelerate their mastering the challenges and produce more road-worth systems soon.

Massive AV and AI Investments

What casual observers may fail to realize is just how many huge corporations are making multibillion dollar investments to get an early adopter advantage. Every major automaker needs AVs. All automakers worldwide are looking for what will work plus the tech firms that are hoping to lock in the next Windows, macOS/iOS, Linux OS or other competitive computer systems for these vehicles.

We may well see a mix of systems as we do with computers now but all must comply with driving rules and regulations—current and planned. There are certainly considerably different driving rules in various countries so one system may not work everywhere.

Large countries, China for instance, has terrible air pollution problems caused primarily by automobiles and power plants. China is really pushing photovoltaic solar panel production and implementation for several reasons: it is a clean non-polluting energy source and their manufacturing and installation expertise opens worldwide market opportunities.

China is making a big push towards electric vehicles (EVs) that will use photovoltaic recharging. If you are already going EV, why not use this transition as the ideal time to automate traffic flow and eliminate the problems of pollution, congestion, gridlock and accidents at the same time?

How serious is China about AI? The country has a development plan to become the world leader in AI by 2030, aiming to surpass its rivals technologically and build a domestic industry worth almost $150 billion.

Smart Connected Cars (SCCs)

NXP Semiconductors, Eindhoven, the Netherlands, and the China Ministry of Industry and Information Technology’s Subsidiary CAICT in Shenzhen recently signed a strategic cooperation agreement on Smart Connected Cars (SCCs). NXP was granted Official Pilot Company Status for Intelligent Transportation and Securely Connected Vehicles in China. CAICT is a subsidiary of China’s Ministry of Industry and Information Technology (MIIT) and the agreement will foster innovation in intelligent transportation and securely connected vehicles. CAICT was appointed by MIIT as project lead for the “Sino-German Intelligent Manufacturing Cooperation program” in 2016.

The CAICT-NXP partnership is focusing on strategic research and development, the development of standards, quality and testing, and talent exchange. It aims to advance China’s car industry with secure connectivity and infrastructure solutions, such as vehicle-to-vehicle and vehicle-to-infrastructure communications for smarter traffic.

The two parties will work on state-of-the-art networking technology and product development, while jointly promoting international standards across automotive applications such as information service terminals, vehicle-to-vehicle communications and vehicle-to-infrastructure communications and other automotive networking applications.

“Every year, 1.3 million people die in road accidents around the globe. The implementation of V2X and other intelligent transport systems will significantly reduce accidents, hours spent in traffic jams and CO2 emissions in China. However, safe and secure mobility can only come to life if there’s a commitment to collaboration. NXP is honored to be appointed as official collaboration company in the pilots, jointly working on this high-impact societal change. We look forward to supporting the transformation of the Chinese automobile industry by providing advanced secure connection and infrastructure solutions for a smarter life,” said Kurt Sievers, NXP Executive Vice President and General Manager of BU Automotive.

Next time: The AV discussion continues with some AI downsides.

Observations & Opportunities: Self-Driving Vehicles—Challenges Abound

Remember when you used the rearview mirror to back into a parking space? Some of us still do but many vehicles now include, or provide as an option, automated parking driving aids. In practice, it conveniently works with another system that avoids backing over unsuspecting pedestrians or hitting inanimate objects (the increasingly common AI-based automated braking systems).

This is just one example of what basic driver-assistance systems can do now. Yes, some consumers are still skeptical of fully AVs (autonomous vehicles)/ADS/ADAS (advanced driver assistance systems). They question their practicality but also like the idea of being able to do other things to pass the time while stalled in traffic congestion. The many heavyweight companies developing AVs are counting on the latter. The expected decrease in traffic accidents and minimized congestion are benefits that insurance companies and governments applaud.

To counter the increasingly common dashboard infotainment systems, and all the mobile devices including laptops for consuming media and information brought onboard, and seemingly ubiquitous smartphones, that dangerously distract drivers now, AVs that could safely transport distracted occupants should become more popular and better for all concerned.

Perhaps the practical limits of AV system implementation are “driver” and passenger sensory overload plus the thought of simply not being in control. Also, significant AV system costs and technology maturity are concerns. Even when AV systems become commonplace, the cost of the components alone will be substantial. As Tesla Inc. indicated recently, the cost differential between assisted driving and full AV capabilities is several thousand dollars. What all of these various systems use is dependent, to varying degrees, upon affordable and reliable artificial intelligence (AI) implementation with associated sensors and actuators. [Also see earlier blogs on AI & Virtual Reality—Let The Games (and Work) Begin and AI & Autonomous Vehicles—Awesome Implications]

Some AV Reality Checks

At this point in time, it would be difficult to find an automobile manufacturer that is not actively working on integrating sophisticated autonomous vehicle operation in some, or all, of its vehicles. At the same time, these manufacturers are increasingly focusing on electric vehicles (EVs) or hybrid gasoline-electric vehicles. In a decade, internal-combustion engine vehicles will be disappearing.

Since many “technology” companies regard the automobile industry as lacking in computer, microelectronics and software expertise, the established computer hardware and software companies are either developing AV systems independently or in joint ventures with familiar automotive companies. Alphabet’s Google, Apple, Intel, Microsoft, and others are very active with massive ongoing investments. Recently, NVIDIA Corp. has been releasing AI chips specifically for AV applications with impressive early evaluations. Realistically, not all will be successfully adopted but differing AV systems will eventually enter mass production.

General Motors Co. recently acquired the LIDAR technology company Strobe, Inc. As part of the deal, Strobe’s engineering talent joins GM’s Cruise Automation team to define and develop next-generation LIDAR solutions for self-driving vehicles. GM thinks its AV team is complete now.

Lidar (also called LIDAR, LiDAR, and LADAR) originally was a surveying technology that measured distance by illuminating a target with a pulsed laser light and measuring the reflected pulses with a sensor. Differences in laser return times and wavelengths made digital 3D-representations of the target.

LIDAR is said to create higher-resolution images that provide more accurate views of the world than cameras or radar alone. As self-driving technology continues to evolve, LIDAR’s accuracy may play a critical role in its deployment.

“The successful deployment of self-driving vehicles will be highly dependent on the availability of LIDAR sensors,” said Julie Schoenfeld, Founder and CEO, Strobe, Inc.

GM is planning to test its vehicles in “fully autonomous mode” in New York state in early 2018, according to New York Governor Andrew Cuomo. However, the planned testing by GM and its self-driving unit, Cruise Automation, will initially be a Level 4 autonomous vehicle.

A level 3 car still needs a steering wheel and a driver who can take over if the car encounters a problem, while level 4 promises driverless features in dedicated lanes. A level 5 vehicle is capable of navigating roads without any driver input and in its purest form would have no steering wheel or brake pedal.

General Motors’ Cruise Automation LIDAR on a Chevrolet Bolt EV (electric vehicle)

 

Caveats

The many AV systems use very sophisticated sensors to evaluate the immediate surroundings that vehicle owners, or automobile mechanics, may not fully understand. Looking at AV going forward, the mainstream vacuum-dependent deposition and etching/cleaning processes will likely need refinements and evolution. Unlike a PC or smartphone problem where failure may be annoying, if a driverless vehicle goes awry because of a microprocessor or sensor component manufacturing defect, the consequences can be tragic and very expensive.

Those using the ICs, MEMS and other micro devices for AI and AV have at least two options to minimize liability, build in considerable redundancy or use fail-safe components and systems. With IC manufacturing, although each device on a wafer may be very similar, they are not absolutely identical. That means that the patterning lithography must be nearly perfect and each layer of material deposited must be incredibly uniform. What was acceptable yesterday may not be good enough going forward.

AI, in the most basic terms, suggests that its (dedicated) computer is cognizant of the specified environment (using sensors) and performs tasks within its defined decision-making capabilities (using actuators). In a vehicle, AV control is tied into an already extremely complex electrical/electronic system with its own central computer and with dedicated embedded computers within specific components—a complex network.

“Artificial intelligence (AI, also machine intelligence, MI) is apparently intelligent behavior by machines, rather than the natural intelligence (NI) of humans and other animals. In computer science AI research is defined as the study of “intelligent agents”: any device that perceives its environment and takes actions that maximize its chance of success at some goal. Colloquially, the term “artificial intelligence” is applied when a machine mimics “cognitive” functions that humans associate with other human minds, such as “learning” and “problem solving” per Wikipedia.

“Intelligent agents are often described schematically as an abstract functional system similar to a computer program. For this reason, intelligent agents are sometimes called abstract intelligent agents (AIA) to distinguish them from their real world implementations as computer systems, biological systems, or organizations.”

Simple reflex agent, Utkarshraj Atmaram, Wikipedia

 

A Broader Perspective of AI and AV

Microelectronics technology is invading many products and usually are interconnected via the Internet. If one looks at a self-driving car, i.e., will the passengers be viewing live TV?

Recently, Gartner, Inc., the Stamford, CT technology research company, highlighted the top strategic technology trends that will impact most organizations in 2018. Creating systems that learn, adapt and potentially act autonomously will be a major battleground for technology vendors through at least 2020. The ability to use AI to enhance decision making, reinvent business models and ecosystems, and remake the customer experience will drive the payoff for digital initiatives through 2025.

“Currently, the use of autonomous vehicles in controlled settings (for example, in farming and mining) is a rapidly growing area of intelligent things. We are likely to see examples of autonomous vehicles on limited, well-defined and controlled roadways by 2022, but general use of autonomous cars will likely require a person in the driver’s seat in case the technology should unexpectedly fail,” said David Cearley, vice president and Gartner Fellow in his Top 10 Strategic Technology Trends for 2018 presentation. “For at least the next five years, we expect that semiautonomous scenarios requiring a driver will dominate. During this time, manufacturers will test the technology more rigorously, and the nontechnology issues such as regulations, legal issues and cultural acceptance will be addressed.”

“AI techniques are evolving rapidly and organizations will need to invest significantly in skills, processes and tools to successfully exploit these techniques and build AI-enhanced systems,” added Mr. Cearley. “Investment areas can include data preparation, integration, algorithm and training methodology selection, and model creation. Multiple constituencies including data scientists, developers and business process owners will need to work together.”

The first three strategic technology trends explore how artificial intelligence (AI) and machine learning are seeping into virtually everything and represent a major battleground for technology providers over the next five years. The next four trends focus on blending the digital and physical worlds to create an immersive, digitally enhanced environment. The last three refer to exploiting connections between an expanding set of people and businesses, as well as devices, content and services to deliver digital business outcomes.

Cearley adds, “Intelligent things are physical things that go beyond the execution of rigid programming models to exploit AI to deliver advanced behaviors and interact more naturally with their surroundings and with people. AI is driving advances for new intelligent things (such as autonomous vehicles, robots and drones) and delivering enhanced capability to many existing things (such as Internet of Things [IoT] connected consumer and industrial systems).”

Next time: Examining widespread ADS/ADAS (advanced driver assistance systems) efforts worldwide continues.

Observations & Opportunities: AI & Autonomous Vehicles—Awesome Implications

Simply put, the business and technology implications for autonomous vehicles, or Automated Driving Systems (ADSs) per the U.S. Department of Transportation (DoT), for those making or using vacuum equipment and related materials, are enormous. These vehicles will require powerful and reliable ICs plus sophisticated MEMS sensors and actuators—all made with vacuum-centric technology to varying degrees.

Will ADSs be widely accepted and their use be on a massive scale? No one seems to know when it will happen on a large scale but some huge global companies are betting that it will be soon. The driving public may be skeptical of Autonomous Vehicles (AVs) at the moment but that was true with personal computers and smartphones too. Before the iPhone just a little over a decade ago, how many of us would have thought that today’s reliance on smartphones, laptops and other mobile devices would be so pervasive, even for TV experiences, on the smaller screens?

Many huge manufacturing and high-tech firms using contract manufacturers to build their products are involved that are not part the automobile industry itself. Insurance companies would really like ADS as would cities where gridlock traffic and accidents are a constant problem as are driver distractions. Adding some control to vehicles could help reduce accidents and traffic jams.

With President Trump proclaiming that massive infrastructure investment is needed, as have several previous presidents, it might actually happen this time. If roads are going to be rebuilt for the 21st Century, then we need smarter roads, bridges, etc. —computational devices, sensors, actuators, precise positioning information, and other technologies to make ADS vehicles safe and reliable. ADSs absolutely rely upon Artificial Intelligence (AI) and that is computational intensive. The value of electrical/electronics content in new vehicles now exceeds the mechanical component costs.

Autonomous Vehicles Defined
So what exactly is an autonomous vehicle or ADS?

“An autonomous car (also known as a driverless car, auto, self-driving car, robotic car) and Unmanned Ground Vehicle is a vehicle that is capable of sensing its environment and navigating without human input. Many such systems are evolving, but as of 2017 no cars permitted on public roads were fully autonomous. They all require a human at the wheel who must be ready to take control at any time.” per Wikipedia [https://en.wikipedia.org/wiki/Autonomous_car]. Intel Corp. uses automotive Advanced Driver Assistance Systems (ADAS) terminology.

Caption: Courtesy of Intel Corporation

 

Why ADS/ADAS?
With so many distracted drivers making your commute or everyday driving trips hazardous, perhaps ADSs does offer some compelling advantages worth consideration.

The U.S. DoT recently issued their “Automated Driving Systems (ADSs)” report. In it, Secretary Elaine L. Chao says, “Today, our country is on the verge of one of the most exciting and important innovations in transportation history— the development of Automated Driving Systems (ADSs), commonly referred to as automated or self-driving vehicles.

“The future of this new technology is so full of promise. It’s a future where vehicles increasingly help drivers avoid crashes. It’s a future where the time spent commuting is dramatically reduced, and where millions more—including the elderly and people with disabilities–gain access to the freedom of the open road. And, especially important, it’s a future where highway fatalities and injuries are significantly reduced.

“Since the Department of Transportation was established in 1966, there have been more than 2.2 million motor- vehicle-related fatalities in the United States. In addition, after decades of decline, motor vehicle fatalities spiked by more than 7.2 percent in 2015, the largest single-year increase since 1966. The major factor in 94 percent of all fatal crashes is human error. So ADSs have the potential to significantly reduce highway fatalities by addressing the root cause of these tragic crashes.”

A report from AAA reveals that the majority of U.S. drivers seek autonomous technologies in their next vehicle, but they continue to fear the fully self-driving car, so far. Despite the prospect that autonomous vehicles will be safer, more efficient and more convenient than their human-driven counterparts, three-quarters of U.S. drivers report feeling afraid to ride in a self-driving car, and only 10 percent report that they’d actually feel safer sharing the roads with driverless vehicles. As automakers press forward in the development of autonomous vehicles, AAA urges the gradual, safe introduction of these technologies to ensure that American drivers are informed, prepared and comfortable with this shift in mobility. [http://newsroom.aaa.com/2017/03/americans-feel-unsafe-sharing-road-fully-self-driving-cars/]

From a practical perspective, it will take a while to improve infrastructure that can accommodate ADSs and for the inevitable tweaking of the many ADS systems.

Tech Giants Pushing ADS Technology

For those exploring ADS technology’s opportunities, consider SAE International’s efforts to date. It is a global association of more than 128,000 engineers and related technical experts in the aerospace, automotive and commercial-vehicle industries [https://www.sae.org/misc/pdfs/automated_driving.pdf]. It’s worth reading their description of then levels of Driving Automation to better understand the challenges involved. Although it certainly addresses vehicles, it also has aerospace implications. Military drones today, and some missiles, use AI to perform their assigned tasks. Decades ago, fully automated aircraft with automated air traffic control (ATC) was demonstrated.

A few weeks ago, Brian Krzanich, the chief executive officer of Intel Corp., announced the fact that Waymo and Intel were collaborating on self-driving car technology.

Autonomous Driving will End Human Driving Errors and Lead to Safer Roads for Everyone.
Per Mr. Krzanich, “One of the big promises of artificial intelligence (AI) is our driverless future. Nearly 1.3 million people die in road crashes worldwide every year – an average 3,287 deaths a day1. Nearly 90 percent of those collisions are caused by human error. Self-driving technology can help prevent these errors by giving autonomous vehicles the capacity to learn from the collective experience of millions of cars – avoiding the mistakes of others and creating a safer driving environment.

“Given the pace at which autonomous driving is coming to life, I fully expect my children’s children will never have to drive a car. That’s an astounding thought: Something almost 90 percent of Americans do every day will end within a generation. With so much life-saving potential, it’s a rapid transformation that Intel is excited to be at the forefront of along with other industry leaders like Waymo.

“Waymo’s newest vehicles, the self-driving Chrysler Pacifica hybrid minivans, feature Intel-based technologies for sensor processing, general compute and connectivity, enabling real-time decisions for full autonomy in city conditions. As Waymo’s self-driving technology becomes smarter and more capable, its high-performance hardware and software will require even more powerful and efficient compute. By working closely with Waymo, Intel can offer Waymo’s fleet of vehicles the advanced processing power required for level 4 and 5 autonomy.

“With 3 million miles of real-world driving, Waymo cars with Intel technology inside have already processed more self-driving car miles than any other autonomous fleet on U.S. roads. Intel’s collaboration with Waymo ensures Intel will continue its leading role in helping realize the promise of autonomous driving and a safer, collision-free future,” added Krzanich

Consumer Adoption of ADS Driving & On-Demand Car Services
The Gartner Consumer Trends in Automotive online survey, conducted from April 2017 through May 2017, polled 1,519 people in the U.S. and Germany, found that 55 percent of respondents will not consider riding in a fully autonomous vehicle, while 71 percent may consider riding in a partially autonomous vehicle. Gartner, Inc. is a global research and advisory company [http://www.gartner.com/technology/home.jsp].

The Gartner survey say that “…concerns around technology failures and security are key reasons why many consumers are cautious about fully autonomous vehicles. Fear of autonomous vehicles getting confused by unexpected situations, safety concerns around equipment and system failures and vehicle and system security are top concerns around using fully autonomous vehicles,” explains Mike Ramsey, research director at Gartner. Survey respondents agreed that fully autonomous vehicles do offer many advantages, including improved fuel economy and a reduced number and severity of crashes. Additional benefits they identified include were having a safe transportation option when drivers are tired and using travel time for entertainment and work.

The survey also found that consumers who currently embrace on-demand car services are more likely to ride in and purchase partially and fully autonomous vehicles. “This signifies that these more evolved users of transportation methods are more open toward the concept of autonomous cars,” said Mr. Ramsey.

Next time: More on widespread ADS/ADAS (advanced driver assistance systems) efforts worldwide.

AI & Virtual Reality—Let The Games (and Work) Begin!

Artificial Intelligence (AI) and Virtual Reality (VR) have enormous potential for entrepreneurs, designers, software experts, video gaming developers and high-tech manufacturers. Some common AI and VR essentials include very powerful computing power in extremely small/tiny spaces and exceptionally high definition displays—real or projected. There will be a requirement for many sensors and actuators too. If today’s marketplace prices are any indication, consumers, businesses and militaries will pay handsomely for the very best. But that presents several challenges including terminology.

AI terminology seems to be described somewhat uniformly but what a given company describes as AI can vary enormously. VR has several common descriptive variants since enhanced reality is more affordable to manufacture and more affordable for consumers. In recent announcements, Microsoft has announced Windows Mixed Reality headsets as part of a new thrust, “We are on a mission to help empower every person and organization on the planet to achieve more, and one of the ways we are doing that is through the power of mixed reality,” said Alex Kipman, Technical Fellow at Microsoft. This is a follow to their earlier HoloLens headsets. The technology behind it will allow its flagship operating system to use the latest generation of Windows 10 hardware devices and software for augmented and virtual reality technologies experiences.

Caption Windows 10 Mixed Reality headsets from partners Lenovo, Acer, Dell & HP

 

Google’s Alphabet’s X division has moved on to refining their technology with the Google Glass 2.0 headsets the result. Google is targeting business and manufacturing applications that will help boost productivity. The original Google Glass headsets had some appeal but some problems that were addressed for the version 2.0 headsets.

Caption: Google Glass 2.0 headset

 

Augmented Reality

“AR is a live direct or indirect view of a physical, real-world environment whose elements are “augmented” by computer-generated or extracted real-world sensory input such as sound, video, graphics or GPS data. It is related to a more general concept called computer-mediated reality, in which a view of reality is modified (possibly even diminished rather than augmented) by a computer. Augmented reality enhances one’s current perception of reality, whereas in contrast, virtual reality replaces the real world with a simulated one.[1][2] Augmentation techniques are typically performed in real time and in semantic context with environmental elements, such as overlaying supplemental information like scores over a live video feed of a sporting event,” according to Wikipedia. [https://en.wikipedia.org/wiki/Augmented_reality]

Apple is excited about Augmented Reality and will soon introduce such capabilities soon on iPhones and iPads. There are rumors that they may intro glasses too. What is for real there is Apple’s ARKit, a set of software developer tools for creating augmented reality apps or iOS.

According to Apple developers, “Apps can use Apple’s augmented reality (AR) technology, ARKit, to deliver immersive, engaging experiences that seamlessly blend realistic virtual objects with the real world. In AR apps, the device’s camera is used to present a live, onscreen view of the physical world. Three-dimensional virtual objects are superimposed over this view, creating the illusion that they actually exist. The user can reorient their device to explore the objects from different angles and, if appropriate for the experience, interact with them using gestures and movement.” With their track record on iOS Apps for iPhones and iPads, that may give them an edge.

Not to be overlooked, however, is the now famous and impressive Oculus Rift virtual reality headset, now a Facebook company. It’s still expensive but very competitive. “Content for the Rift is developed using the Oculus PC SDK, a free proprietary SDK available for Microsoft Windows (OSX and Linux support is planned for the future).[62] This is a feature complete SDK which handles for the developer the various aspects of making virtual reality content, such as the optical distortion and advanced rendering techniques”, according to Wikipedia.

With Augmented Reality (AR), one can immerse themselves in games, education, work, movies or whatever. The big question, however, is will this be just a short-term fad or will it be enduring like the personal computer or smartphone? We just don’t know yet but the question of industry-wide standards versus proprietary platforms will likely be an issue. We do know that familiar tech giants are betting on VR and AR for current and future products. What is certain is that thin-film technologies are essential for manufacturing the LCD, OLED or other screens used in the requisite headsets. If you thought that building 4K or 8K HDTVs was challenging, VR and ER in small form factors will make it interesting. The VR headset options abound.

A Crowded Market

“By 2016 there were at least 230 companies developing VR-related products. Facebook has 400 employees focused on VR development; Google, Apple, Amazon, Microsoft, Sony and Samsung all had dedicated AR and VR groups. Dynamic binaural audio was common to most headsets released that year. However, haptic interfaces were not well developed, and most hardware packages incorporated button-operated handsets for touch-based interactivity. Visually, displays were still of a low-enough resolution and frame-rate that images were still identifiable as virtual. On April 5, 2016, HTC shipped its first units of the HTC VIVE SteamVR headset. This marked the first major commercial release of sensor-based tracking, allowing for free movement of users within a defined space. “ per Wikipedia [https://en.wikipedia.org/wiki/Virtual_reality_headset].

Is AI Ready?

“We are at an inflection point in the development and application of AI technologies,” according to the Partnership on AI [https://www.partnershiponai.org/introduction/]. “The upswing in AI competencies, fueled by data, computation, and advances in algorithms for machine learning, perception, planning, and natural language, promise great value to people and society.

“However, with successes come new concerns and challenges based on the effects of those technologies on people’s lives. These concerns include the safety and trustworthiness of AI technologies, the fairness and transparency of systems, and the intentional as well as inadvertent influences of AI on people and society.

“On another front, while AI promises new capabilities and efficiencies, the advent of these new technologies has raised understandable questions about potential disruptions to the nature and distribution of jobs. While there is broad agreement that AI advances are poised to generate great wealth, it remains uncertain how that wealth will be shared broadly. We do, however, also believe that there will be great opportunities to harness AI methods to solve important societal challenges.

“We designed the Partnership on AI, in part, so that we can invest more attention and effort on harnessing AI to contribute to solutions for some of humanity’s most challenging problems, including making advances in health and wellbeing, transportation, education, and the sciences.”

Some founding members of this Partnership on AI include Apple Inc., Amazon, DeepMind, Facebook, Google (Android, Chrome), IBM (Watson computer), Intel Corp., Microsoft, Sony (PlayStation) and the The Association for the Advancement of Artificial Intelligence (AAAI).

What Is Needed

In future blogs, we’ll revisit some specific AI and VR applications of interest and challenges.

Observations & Opportunities

Faster Changes and the Implications

We are indeed in the midst of rapidly changing times. Some of today’s global issues and challenges are the result of using various technologies without thinking the impact through. Regardless, science and technology will be essential for solving these problems.

Something to keep in mind while contemplating tackling anything new with science and technology. It is prudent to consider the impact of any new product or manufacturing process itself, all of the equipment and materials involved, and what happens throughout the world as advanced technologies permeate virtually everything in our business or personal lives.

What’s Really New

We all have some readily available topics in mind when “new” technologies are mentioned but we probably overlook several areas of great potential. Below is the thought-provoking Hype Cycle for Emerging Technologies, 2017 chart from Gartner Inc. Every topic on the chart is significant and there are overlapping interactions between these new technologies that are likely unpredictable. Someone invariably uses a product or technology for something totally overlooked by the original technology inventors and proponents—and unintended consequences result.

Hype Cycle for Emerging Technologies, 2017. Source: Gartner July 2017

Gartner, Inc., of Stamford, Connecticut, is a global research and advisory company. It helps business leaders in every industry and enterprise with some objective insights needed to make the right decisions with analyses and predictions that can prove insightful.

As Gartner states in its “Gartner Identifies Three Megatrends That Will Drive Digital Business Into the Next Decade” [http://www.gartner.com/newsroom/id/3784363], “The emerging technologies on the Gartner Inc. Hype Cycle for Emerging Technologies, 2017 reveal three distinct megatrends that will enable businesses to survive and thrive in the digital economy over the next five to 10 years. According to Mike J. Walker, research director at Gartner, “Artificial intelligence (AI) everywhere, transparently immersive experiences and digital platforms are the trends that will provide unrivaled intelligence, create profoundly new experiences and offer platforms that allow organizations to connect with new business ecosystems.”

The Hype Cycle for Emerging Technologies report is the longest-running annual Gartner Hype Cycle, providing a cross-industry perspective on the technologies and trends that business strategists, chief innovation officers, R&D leaders, entrepreneurs, global market developers and emerging-technology teams should consider in developing emerging-technology portfolios.

“The Emerging Technologies Hype Cycle is unique among most Gartner Hype Cycles because it garners insights from more than 2,000 technologies into a succinct set of compelling emerging technologies and trends. This Hype Cycle specifically focuses on the set of technologies that is showing promise in delivering a high degree of competitive advantage over the next five to 10 years,” added Walker.

Their Hype Cycle graphic, above, touches upon many exciting technologies including IoT (Internet of Things), autonomous vehicles, nanotube electronics and many other key categories. The obvious thread that ties all of these things together is ubiquitous computing and communications access plus advanced software. Gartner’s take on when these technologies reach mainstream adoption seems reasonable today but unforeseen events, or just some major company’s very secret product launch plans, could significantly alter those predictions.

Just recall Apple Inc.’s launch of the iPhone a decade ago. We already had our cell phones but Apple’s elegant design plus marketing spin on what consumers wanted, or could be convinced that they needed and could use, proved remarkable. The iPhone changed mobile information access and use in profound ways as well as computing in general.

Today’s plethora of smart phones, tablets and laptops are all influenced by the iPhone to varying degrees. With numerous companies like Apple and Google pursuing Augmented Reality (AR) and/or Virtual Reality (VR) to make information access and usage better while improving work or entertainment experiences, these technologies can have both predicted and unintended consequences too.

The Internet is essential now for most businesses and consumers which has driven its enormous growth. However, Internet security concerns are something that requires ongoing vigilance to avoid viruses, malware and scams to protect its viability. It also requires products with built-in protection and the ability to upgrade protection as new threats emerge.

AI or VR features are already appearing in more consumer products as well as the Internet of Things (IoT) for consumers and businesses as well. VR and ER plus more powerful personal computers, smartphones and tablets are certainly changing how many work and play. Not everyone will need and/or use all of these things but the impact will be felt by everyone. If you want to check the weather, you probably get more useful information quicker on your smartphone or tablet than on your PC.

A pixel comparison chart of SD (standard definition), Full HD (1080p), 4K Ultra HD and 8K Ultra HD displays. Image by By Libron – Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=25976260

Whether it is your laptop, desktop, smartphone, tablet, 4K or soon 8K TV, automobile dashboard, thermostat or any other product with an electronic display, it is obvious that display technology continues to advance too. Many were content with HDTV (1080p) until the 4K TVs appeared and the better pictures combined with a marketing push made 4K Ultra HD TV a must-have product for many people. Soon the 8K push will begin as soon as enough cameras are available for content production, movies at first and TV later. Of course, Blu-ray and other disc player manufacturers will have to add upconversion for the 8K sets. Since high-speed Internet is still not available everywhere, discs will prove useful for a while.

AI technologies will be the most disruptive class of technologies over the next 10 years due to radically increased computational power, near-endless amounts of data, and unprecedented advances in deep neural networks; these will enable organizations and governments with AI technologies to harness data in order to adapt to new situations and solve problems that no one has ever encountered previously.

Walker believes that enterprises seeking leverage in this theme should consider the following technologies: Deep Learning, Deep Reinforcement Learning, Artificial General Intelligence, Autonomous Vehicles, Cognitive Computing, Commercial UAVs (Drones), Conversational User Interfaces, Enterprise Taxonomy and Ontology Management, Machine Learning, Smart Dust, Smart Robots and Smart Workspace. This will require some serious homework to grasp the implications and the best ways to use it. These new tools may raise concerns about George Orwell’s novel Nineteen Eighty-Four becoming real by some observers. Of course the novel was just fiction.

Note: Upcoming weblogs will address why MEMS devices are essential in mobile devices and many other products and why vacuum technology is essential for their manufacturing.

Vacuum Observations & Perspectives

In this weblog series, we’ll examine some emerging and evolving opportunities for vacuum-centric equipment, materials, processes and R&D. Also, we’ll look at some interesting applications and technology trends. Thinking outside the box, if you will, with a broad perspective.

Vacuum use is only likely to increase going forward, especially when producing very small and complex components like ICs and MEMS for sophisticated products. Doing so in a vacuum is the only practical way to prevent gaseous and particulate contamination. Vacuum is absolutely essential for semiconductor production.

Semiconductor Challenges—Manufacturing Is Difficult

The familiar semiconductor industry is changing, significantly, now. These changes provide many opportunities for those companies involved that can meet the challenging technical and investment demands. The microchip industry itself is always continuing its quest for ever smaller geometries (circuit features) to get more devices produced per wafer area. That’s how Moore’s Law has keep innovation and performance increases alive.

By constantly pushing new frontiers of equipment, materials, and processes, chipmakers produce the ubiquitous integrated circuit (IC) chips at reasonable prices. Historically, jumps in wafer size were tied to photolithography advances that enabled smaller feature sizes. The result was more advanced semiconductor chips with greater performance. Transistors are much smaller now and circuits more complex but greater performance results. But they are extremely difficult to manufacture.

The greatest demand for computer chips now is in mobile devices, wireless, IoT and automotive areas. Laptops, tablets and smartphones all benefit. Since desktop PC performance is already greater than what most customers actually need, mobile devices are often a primary focus. For products where space and extreme performance are not necessary, conventional IC manufacturing fabs will be cranking out those chips for years. You certainly don’t need state-of-the-art microprocessors in your home’s thermostat.

Monolithic ICs

Semiconductor manufacturing, often considered a mature industry, needs near-perfect thin films and truly advanced lithography patterning along with sophisticated deposition and etching process technologies to produce the active and passive elements in ICs—the core of most high tech products worldwide. Planar 2D production is mainstream now in semiconductor manufacturing but some high-profit, in-demand chips are now made with 3D circuitry. Conventional optical photolithography now appears to be at the end of its affordable smaller dimensions potential but there is a promising new exposure system: EUVL.

2D, 3D & EUV Lithography

To produce more chips in the same wafer surface space, you need smaller chips that provide the functionality but in smaller dimensions. Greater functionality and faster performance are the norms. If the latest EUVL (Extreme UltraViolet Lithography) tools prove ready for prime time with high-volume production next year, the quality and flatness of the many deposited thin film layers will become even more critical than they are now. EUV needs a stringent vacuum environment. See “THE USE OF EUV LITHOGRAPHY IN CONSUMER MICROCHIP MANUFACTURING” at http://www.pitt.edu/~budny/papers/23.pdf for some insights.

In July 2017, ASML Holding N.V. (ASML) President and Chief Executive Officer Peter Wennink said, “In EUV lithography, we have integrated an upgraded EUV source into a TWINSCAN NXE:3400B lithography system in our Veldhoven [The Netherlands] facility and achieved the throughput specification of 125 wafers per hour on this system. Now, with all key performance specifications demonstrated, we focus on achieving the availability that is required for high-volume manufacturing as well as further improving productivity.”

That sounds promising but the actual EUV masks and photoresists seem somewhat problematic from published comments and reports. In July 2017, BACUS (formerly the Bay Area Chrome Users Society) noted, “Recently, readiness of the EUVL infrastructure for the high volume manufacturing (HVM) has been accelerated [1]. EUV source availability, the first showstopper against EUVL HVM, has been dramatically increased and close to the targets for HVM insertion. Mask defectivity, another focus area for the HVM, has also been concerned. Due to the difference in mask and optics appropriate for the wavelength between EUV and ArF lithography, specialized metrology tools are required in EUVL. However, current DUV and e-beam inspection tools are easy to miss the printable phase defects in EUV mask since the lights of corresponding wavelengths cannot penetrate multilayers (MLs)[2,3]. Therefore, the actinic review system is essential to provide defect free EUV masks.” See more details at https://spie.org/Documents/Membership/BacusNewsletters/BACUS-Newsletter-July-2017.pdf

The Samsung R&D experts who authored this are betting on EUV and 7nm lithography to take some IC foundry business away from the leading foundry producer TSMC (Taiwan Semiconductor Manufacturing Co.) but TSMC is also planning on introducing 7nm EUV devices next year. We’ll see. EUV keeps getting promised as “soon” but the dates keep slipping. Reuters noted, “But the firm lags well behind Taiwan’s TSMC in contract manufacturing: TSMC held a market share of 50.6 percent last year compared with Samsung’s 7.9 percent, according to research firm IHS. It also trailed U.S.-based Global Foundries, which had a 9.6 percent share, and Taiwan-based UMC’s 8.1 percent.”

Wafer Sizes Matter

First, there are wafer size considerations. Today, 200mm wafer fabs are typically running at capacity with some new 200mm fabs being built but some essential 200mm fabrication production tools are in short supply and expensive. Some of these fabs once made high-volume state-of-the-art ICs but have transitioned to more profitable proprietary and/or lower volume chips. Supporting 200mm will be necessary for the foreseeable future.

Per Christian G. Dieseldorff, Industry Research & Statistics Group, SEMI, at SEMICON West 2017, “Driven by mobile and wireless applications, IoT (Internet of Things), and automotive, the 200mm market is thriving.  Many of the products used in these applications are produced on 200mm wafers, so companies are expanding capacity in their facilities to the limit, and there are nine new 200mm facilities in the pipeline. Looking only at IC volume fabs, the report shows 188 fabs in production in 2016 and expanding to 197 fabs by 2021. China will add most of the 200mm capacity through 2021, with 34 percent growth rate from 2017 to 2021, followed by South East Asia with 29 percent and the Americas with 12 percent.”

The 300mm fabs, once expected to displace 200mm fabs, are now competing with 200mm in some markets where smaller volumes can make 300mm efforts more expensive. But 300mm is running at capacity too with the most advanced chips. There also has been the realization by the major semiconductor manufacturers that many improvements to optimize 300mm manufacturing are possible which delays the costly transition to building factories that handle 450mm wafers.

Finally, 450mm seems to be a wafer size that can wait, perhaps until 2020 per SEMI [http://www.semi.org/en/node/50856]. Also, the New York-based 450mm global consortium, G450C, is defunct. Samsung, and others, are now stacking many layers of transistors on the same memory die with smaller transistors, so the need for 450mm wafers is not as urgent now although companies are still exploring 450mm options for the future.

Then, of course, there is the high-end leading edge efforts to switch to EUV lithography in the quest to produce even more ICs per unit area on a wafer. EUV emerged when x-ray proved problematic years ago. X-ray lithography was first proposed by H. Smith and Spears at MIT. [https://www.researchgate.net/publication/299496830_X-ray_Lithography_Some_History_Current_Status_and_Future_Prospects]. There are recollections of the frantic x-ray lithography efforts several years ago that never reached mainstream production status. X-ray lithography promised > 1nm feature sizes, far smaller than the EUV efforts of today will produce.

Mainstream x-ray lithography simply had too many issues at the time: dangerous x-ray sources as well as expensive masks and resists that were problematic. Proposed x-ray synchrotron radiation sources required very long times to reach acceptable low vacuum levels which is problematic for volume production lines that cannot stop operating for maintenance. For some perspective on x-ray lithography’s origins, check out “X-ray lithography: Some history, current status and future prospects” by Juan R. Maldonado and Martin Peckerarat https://www.researchgate.net/publication/299496830_X-ray_Lithography_Some_History_Current_Status_and_Future_Prospects.

Note: Upcoming weblogs will address IC lithography issues and why MEMS devices are essential in mobile devices and many other products.