What is augmented reality? A beginner’s guide

Have you ever used a Snapchat Lens or played Pokémon GO? If so, you’ve already experienced an Augmented Reality (AR) application—and you’re likely to experience many more in the near future.

AR describes technology designed to enhance or add to a user’s experience of reality. This rapidly developing technology has countless potential applications, and there’s never been a better time to delve into the field.

This article will explain what AR is and how it’s used. We’ll also introduce some of the key technologies and tools that aspiring AR developers need to know.

Let’s get started!

We’ll cover:

What is AR?
How is AR used?
How does AR work?
What should I learn to become an AR developer?
Wrapping up and next steps

What is AR?

To “augment” something is to make it greater by adding to it, and that’s exactly what AR applications do. They enhance part of the user’s physical reality with computer-generated sensory input. The most common modalities for this input are visual, auditory, and haptic (relating to touch or proprioception), but other modalities are also possible.

AR is part of a broader field known as Extended Reality (XR). XR refers to all technologies that combine the real and digital worlds to create a new kind of experience. It’s an umbrella term that includes AR, Mixed Reality (MR), and Virtual Reality (VR).

The lines between these subfields of XR aren’t always clear, but in general, here’s how they break down:

Augmented Reality: Digital elements appear over representations of the real world, with limited interactivity between them. AR often takes place via smartphone. Example: In the smartphone game Pokémon GO, players hunt for characters that appear on their screens, superimposed over images of the real world.
Mixed Reality: Digital elements are anchored to representations of the real world to create opportunities for interactivity. Example: Microsoft’s HoloLens headset lets users interact with 3D holographic images as though they are part of the real environment.
Virtual Reality: The user is immersed in a fully digital, fully interactive environment. Example: Sony’s PlayStation VR headset immerses users in virtual reality games.

How is AR used?

In recent years, technological advances have created explosive growth in the field of AR. AR may soon touch nearly every economic sector and become part of everyday life for many of us.

Here are just a few of the current and potential use cases for AR:

Entertainment: Make video chats more fun with stickers, lenses, and games
Shopping: Try on clothes virtually or visualize furniture in your home before purchasing
Sports: Draw lines on sports fields to illustrate and analyze plays
Navigation: Use a GPS overlay atop the real environment to provide directions
Medicine: Create projections of the brain to assist with neurosurgery
Military: Project altitude, speed, and other data onto the helmet visors of fighter pilots to enhance focus
Logistics: Deliver information about cargo containers to crew members to speed up loading times
Museums: Bring exhibits to life by providing narration, a representation of the artist, or other extra information
Archaeology: Project views of ancient civilizations over today’s ruins to aid in research

How does AR work?

AR relies on several advanced technologies, including specialized hardware and software. Exactly how an AR application works depends on the sensory modality used for the computer-generated input.

Visual input is the most common AR modality, so here’s a basic outline of how it works:

Sensors collect data about the environment. In smartphones, this data comes from the camera, accelerometer, and gyroscope. Some AR glasses also have built-in time-of-flight depth sensors for better 3D imaging.
A software algorithm creates a virtual map of the environment. This software draws heavily on the fields of computer vision and machine learning.
The algorithm locates the device within the virtual map. This process is known as tracking (also called localization). Tracking happens many times per second, so the application always knows where the user is relative to the environment.
Digital content is placed within the virtual map. Since the virtual map is invisible, the content looks like it’s part of the physical world. At this point, the device’s sensors may collect subtle information about the physical environment (like light intensity and color temperature). The application can then project those qualities onto the digital content to make it look more realistic.

What should I learn to become an AR developer?

It’s an exciting time to enter the field of AR. Refinements in technology are constantly improving the quality of AR experiences, and new industries are beginning to seek out AR solutions.

So what should you learn to become an AR app developer? The specific answer depends on what kinds of projects you’d like to work on. That being said, here are some core skills that would benefit any AR developer.

Programming languages

AR developers require strong programming skills. Whether starting from scratch or transitioning from another field, focusing on languages commonly used in AR development makes sense.

The most common languages vary by platform and include:

Apple (iOS): C/C++, C#, or Swift
Google (Android): Java

JavaScript is another helpful language, particularly if you plan to create web apps.

AR SDKs

As an AR app developer, you don’t necessarily need to know all the ins and outs of computer vision algorithms (unless you plan to get involved in R&D). An AR Software Development Kit (SDK) can handle this task and many others—no need to reinvent the wheel.

AR SDKs are full-featured toolkits for creating AR experiences. The job of the AR SDK is to fuse digital content with the real world. To that end, AR SDKs contain libraries that handle complex tasks like tracking, object recognition, light estimation, content rendering, etc.

Both Apple (iOS) and Google (Android) offer AR SDKs specific to their mobile platforms. Apple’s SDK is called ARKit, while Google’s is called ARCore. Other well-known AR SDKs include Wikitude and Vuforia.

3D modeling and game engines

At some point, you’ll need to create the actual digital content you’ll be placing in your AR apps (e.g., the Pokémon!). This requires 3D modeling skills.

3D modeling is a vast field in itself, and there are many tools to choose from. Most professional AR developers work with popular game engines such as Unity (which uses the C# programming language) and Unreal Engine (which uses C++).

UI/UX skills

AR apps succeed or fail based on their ability to engage users and deliver enjoyable experiences. You need to be able to put yourself in the user’s shoes for your app to have the intended effect. Therefore, a UX or UI design background is a valuable asset for any AR developer.

Wrapping up and next steps

The potential of AR seems almost limitless, and we’ve only just scratched the surface. If you’re excited about this growing field, we encourage you to start investing in the skills needed for success. The career opportunities presented by AR will only continue to grow.

Your learning path will depend on your current skill level and what kinds of projects you hope to do in the future. Looking for a place to get started? The courses and paths from Educative listed below would both make great jumping-off points.

Happy learning!