Designing for the future: The Evolution of App and Device Design for Augmented Reality

How can Augmented Reality influence the future of both app design and device design?

With the recent launch of Apple’s ARKit SDK (Software Development Kit) with iOS 11, Augmented Reality is starting to emerge as an exciting new tool on modern mobile devices, meaning it will have a greater presence moving forward. For such technology to become as successful as predicted, both app design and device design are beginning to evolve to account for Augmented Reality technology.

As Tim Cook (Apple CEO) said in an interview with David Phelan from the Independent in February 2017: 

"I view AR like I view the silicon here in my iPhone... It's not a product per se — it's a core technology." 

Both app design and device design are already beginning to accommodate Augmented Reality technology

First thing first…what is Augmented Reality?

Many of you may be familiar with the term; these days it’s used to describe the technology that allows you to chase Pokémon with your phone, or transform yourself into a dancing cat thanks to Snapchat’s ‘entertaining’ filters. But in the most basic terms, Augmented Reality (also known as AR) is the overlay of virtual, digital content onto the real-world.

How does it work?

Augmented overlay is triggered by a process of recognition and tracking. Following the example of Snapchat, the filters appear on screen as soon as a face has been recognized by the phone or tablet’s camera. This is an example of object or face tracking. There are three other means of recognition that prompt AR technology: GPS tracking has been used in apps such as Pokémon Go, where the virtual overlay is prompted to appear on screen when a device reaches or recognizes certain GPS coordinates that have been pre-embedded within the system. This is similar to location tracking, whereby the camera recognizes the location before displaying the virtual object on screen. This can be particularly beneficial to architects or property developers; AR can overlay a computer-generated building to show what it will look like in-situ. The fourth and final method is image or target based tracking and is perhaps the method most familiar or recognizable when thinking about AR. Unlike the previous three methods (which can be branded as 'markerless' augmented reality), target based AR relies on image recognition which is encoded with the virtual image. When the camera recognizes the unique marker, the virtual content is displayed onscreen. 

Found them yet? 'Pokemon Go' uses GPS based AR 

Industry talk – Where to next?

Although Augmented Reality has been around for quite a few years, the technology is still in it’s infancy. The potential for AR to grow and become fully absorbed into everyday life is possible, but there is still much more development and research needed until we reach this this point. For now, AR companies and developers are expected to continue to operate on a B2B basis, expanding more into sectors such as gaming, PR & marketing, education, and industry. There are already examples of successful AR applications that have been developed within these industries: CGEye's CBS app being one. The AR based application has been proved a successful sales tool for CBS, a company that manufactures ergonomic computer arms. Instead of lugging in their heavy products to a sales pitch, they are now able to show ergonomic equipment digitally via the app. 

"CBS are the ideal candidate for AR technology. They are forward-thinking, adventurous and innovative in their thinking. We opened up a discussion with them to find a way for their sales agents to be able to demonstrate the CBS range of monitor arms in a slick, convenient and engaging way." - Paul, Company Director of CGEye Ltd and Rise AR

Below is a video demonstrating the app:

So how does AR effect the future of both app and device design?

For Augmented Reality to be as successful as it has the potential to be, both app design and phone design needs to develop and evolve to accommodate AR technology.

One of the main considerations, and perhaps challenges, when designing an AR based application is that for AR to function the screen needs to have little or no other clutter. In other words, app design needs to evolve to allow a greater screen real-estate for better use and a better experience of augmented reality. This becomes most beneficial on a smaller screen, such as a mobile device, where the technology is expanding and where existing screen space is minimal.

App design will also have to consider the increase of wearable devices such as Daqri’s Mixed Reality (MR) headset, that is described as “a visionary tool for the 21^st century worker”, or Microsoft’s HoloLens. The most likely adopters for this brand of AR are heavy industry and education. Rather than displaying virtual content on a flat mobile or tablet screen, an AR based app for a headset needs to be compatible over a larger surface area and account for a greater field of vision to allow for safe and successful use within the workplace.

The above examples of Daqri and HoloLens show how wearable device AR technology is beginning to emerge within industry, but unfortunately it will still be a while until wearable devices compatible with augmented reality reaches the mainstream. This is partly down to cost; for such technology to emerge within the consumer market, the price of wearable devices needs to decrease significantly. But it is also down to design; it is unlikely anyone would choose to wear a bulky headset walking down the street – the future of wearable devices lies partly with a sleeker design. It will need to address practicality issues while also increasing aesthetic value. For now, the technology is still fairly inaccessible: it will be a while before you can say goodbye to your phones and access your messages via augmented reality wearable technology.

In terms of phone design, the modern phone is pretty much guaranteed to come complete with a front and back camera. This already makes AR much more accessible to the consumer; without a front camera the likes of Snapchat would not have been able to develop their much-loved filters - oh how the world would be a different place! As mentioned above, a camera fundamentally allows AR technology to work; no camera means no AR! However, to develop and bring to the consumer more advanced and more complex augmented reality solutions, AR would require a dual-camera system. This has already been introduced as a feature to the iPhone 7 Plus, and will almost certainly be a feature on the much-anticipated iPhone 8 that is expected to be released in Autumn of this year. The benefit of a dual-camera is that it increases tracking accuracy; software can triangulate distances to detect corresponding points in the two images. As a result, we would see less marker based AR systems, and an increase in 'markerless' AR. If augmented reality is to become accessible to the consumer market going forward, a dual-camera system would be a beneficial feature within future mobile devices. 

The Dual-Camera opens up more complex AR technology to the masses - a feature beneficial in future mobile and tablet design

This blog post is a part of Design Blogger Competition organized by CGTrader. 

For more info, visit: https://www.cgtrader.com/blogging-contest#section-awards

 #designingforfuture