Augmented Reality, Apple’s Been Tinkering Since 2010

A/R and V/R (Virtual Reality) technologies have gathered quite a momentum over the last few years. Many analysts have criticized Apple’s lack of initiatives in this space. The watch and wait strategy on part of Apple has also build angst among its fans when it comes to innovation.

In early October this year, Tim Cook spend some time discussing his elaborate opinion on Augmented Reality during a tech event in Utah. In his comments, he said that AR is poised to become an integral part of our every day experiences. Here is an excerpt from that talk:

“I do think that a significant portion of the population of developed countries. And eventually all countries, will have AR experiences every day. Almost like eating three meals a day, it will become that much a part of you. A lot of us live on our smartphones, the iPhone, I hope, is very important for everyone. So AR will become really big. VR I think is not going to be that big, compared to AR. I’m not saying it’s not important, it is important.”

The Latest Apple Patent–and it’s Augmented Reality!

This morning, the US Patent Office approved Apple’s patent, 9,488,488 . This Apple Patent titled Augmented Reality Maps gives us a taste around Apple’s early thinking around mobile-based augmented reality capabilities.

What is more interesting is that the patent at hand was filed way back in Feb, 2010 by inventor Jaron Waldman. Jaron led the Geo based services offering at Apple between 2009 and 2013 and currently is the co-founder and CEO at Curbside inc, Palo alto.

This would signify that contrary to the popular opinion, Apple has in fact been working on mobile Augmented reality for quite some time. (last 6 years or more!)

The thinking around the Augmented reality Maps patent is to use iPhone  to provide more value to the user during navigation.

The Apple Augmented Reality Proposition

According to the patent, the proposed A/R technology can take advantage of a iPhone or iPad’s imaging and display capabilities and combine a video feed with data describing objects in the video. In some examples, the data describing the objects in the video can be the result of a search for nearby points of interest.

For example, a user visiting a foreign city can point an idevice and capture a video stream of a particular view. A user can also enter a search term, such as museums and get a A/R view of the nearby museums of interest.

The whole idea of combining A/R and Map Navigation breaks down into the four key steps:

In practice, a user points an iphone to capture and display a real-time video stream of a view. The handheld communication device detects a geographic position, camera direction, and tilt of the image capture device.

The user sends a search request to a server for nearby points of interest. The handheld communication device receives search results based on the search request, geographic position, camera direction, and tilt of the handheld communication device.

The handheld communication device visually augments the captured video stream with data related to each point of interest. The user then selects a point of interest to visit.

The handheld communication device visually augments the captured video stream with a directional map to a selected point of interest in response to the user input.

I think that the process thinking around Augmented reality has been there at Apple for quite some time. They have been tinkering around ideas that can help the company provide real and tangible user benefits via these new technologies.

In the past, the iPhone was perhaps not powerful enough to do the heavy processing that is required to render A/R applications. This is changing. The new A 10 Fusion processor is remarkably more faster and powerful that previous iPhone processors.

There is a delicate balance between battery use and processing power. Samsung learned the lesson the hard way.

I think that with the next generation iPhone, likely releasing next year marking its 10th anniversary, we are going to see some awesome applications of mobile A/R.  Applications that actually bring tangible value and balance it with the battery life and processor performance.

Until then, it’s pretty much a guessing game as with any other Apple Patent.