Introducing Augmented Reality in the Aerospace Industry

Augmented reality (AR) might be fashionable in the current market, but it is already well-established in the military aircraft sector. “It’s been around for nearly 60 years,” states Chris Colston, director of strategic growth at BAE Systems, which manufactured the earliest head-up display (HUD) for the Blackburn “Buccaneer” aircraft, which launched in the late 1950s. “We’ve supplied AR solutions long before that meant anything to the mass market.”¹

In recent years, HUD, AR, and virtual reality (VR) systems have become commonplace in the aerospace and defense industries, with applications encompassing manufacturing quality control, engineer and pilot training, intelligence and information communication in combat operations. Notably, AR is being utilized inside the cockpit to incorporate digital information with reality as a means of enhancing pilot awareness and safety.

Aircraft Augmented Display Types

Among military aircraft, helmets typically supply information via AR displays, integrated with the instrumentation, sensing, and camera systems of the aircraft for which they have been designed. For instance, the new Striker^® II Digital Helmet-Mounted Display from BAE systems offers 3D audio, night vision and target tracking for fighter jets.

HUDs, which project information onto transparent screens in the pilot’s line of sight, usually onto a windshield or visor, represent an additional class of augmented display typically found in both military and commercial aircraft.

Cockpit of a military aircraft with Thales head-up display shown mounted above the control panel. Image Source: Thales Group

The most recent class of augmented visual systems for pilots shares physical characteristics with consumer AR smart glasses. For instance, San Diego-based Aero Glass is offering a system integrating smart glasses and overlay panel to offer 3D, 360-degree AR capabilities to all pilots.

Engineers at BAE are even researching the futuristic notion of a “wearable cockpit”, whereby a pilot’s helmet operates as a comprehensive personal avionics suite. It would seem that technology is not so far away from realizing the scene in Marvel’s Black Panther movie, in which ex-Air-Force-fighter-pilot-turned-CIA-operative Agent Ross “flies” a mission to pursue and shoot down enemy planes all from a virtual cockpit.

Cockpit AR Design Considerations

In the contemporary aviation industry, visual clarity is essential for all categories of augmented cockpit displays, irrespective of the technology or form factor. If information presented on a HUD screen, AR device, or helmet visor impede the pilot’s capacity for seeing and quickly comprehending the real-life exterior environment, then the technology will do more harm than good.

Because an AR projection is viewed on a transparent screen with the operator’s surroundings visible to the rear, any letters, markings, and symbology need to contrast particularly well with the background environment. This necessitates accurate luminance and color settings, which dynamically adapt as the environment adapts or ambient lighting conditions are adjusted.

An additional important consideration is the projection focal point. If a pilot needs to follow a distant horizon or scan features in the landscape, shifting focus to observe information on a display screen located within the cockpit can be tiring and distracting. Conversely, projecting augmented information in the pilot’s line of sight—at variable distances more accurately matching the depth of the environment—can save that pilot the energy of regularly refocusing.

Ensuring AR Display Quality

Designers and manufacturers of AR displays need to satisfy visual performance criteria for color, contrast, resolution, brightness, and focus. Information should be exhibited clearly and consistently, regardless of ambient lighting situations and operating conditions. Guaranteeing the correct functioning of AR display screens, helmets, and smart glasses necessitates precise measurement and testing during both the R&D and production phases. Radiant offers comprehensive camera-and-software solutions for evaluating the quality of near-eye displays (NEDs), such as AR/VR devices and HUDs, that are viewed near to the human eye.

Acknowledgments

Produced from materials originally authored by Anne Corning from Radiant Vision Systems.

References and Further Reading

1. As quoted in Pozniak, H., “Augmented reality: making sci-fi true for the modern military,” The Telegraph, June 27, 2019
2. Savvides, L., “Hey Siri, take off! Get ready for more-advanced planes”, CNet, March 2, 2018

This information has been sourced, reviewed and adapted from materials provided by Radiant Vision Systems.

For more information on this source, please visit Radiant Vision Systems.