Finding the right use for AR technology and to bring it to customers to solve problems they may encounter

As part of the Design Lab, I worked on a hypothetical project that took 120 hours with a 6-week period.

Heem, a hypothetical client, is a new interior design app that uses augmented reality (AR). Heem has a partnership with the top ten furniture stores in the U.S. and has scanned all of the items carried by these stores. Heem's ultimate goal for this project was to find the right use for AR technology and to bring it to customers to solve problems they may encounter or to provide additional value to their current goals.

Client:

Team:

Timeline:

Tools:

Heem

Independent work with the help of a mentor

120 hours, 6 weeks

Sketch, InVision, Maze, Illustrator, Photoshop

How Users React The AR?

Since AR is new for most people, I was curious about how do they react to AR. Through observation and note-taking, I asked participants to try other apps that use AR, such as IKEA Place, Wayfaire, and CB2. I observed several common behaviors, and made a note about how they react.

Common Behaviors

It seems that the initial instructions on those apps (e.g., "Tilt your device toward a flat surface," "Point your device at a surface," etc.) are confusing. For the participants, it was the first time using any AR app, so they tend to move the device more than needed because the instructions didn’t mention how much or how far they should orient the device.
Once a 3D item showed up on the screen, they were impressed.
They moved around the room to see the item. The items they picked were too big for their smartphones’ screens to see the item without moving.
They moved the device right and left to see the full item.

Feedback

Positive

The 3D looks real
The 3D items show all details in 360 degrees and show cable holes on the backside of the TV cabinet

"I can try the items in my space."

"It is great that I can see how big/small the item is in my space."

Negative

The 3D item looks floating because there was no shadow under the item
The 3D item went over the spot where supposed to be a wall
Dark-colored floors were hard to recognize for the app

"It's hard to see the depth." (translated from Japanese)

"I don't trust the item fits my space because the 3D went over the spot where supposed to be a wall."

A screenshot of Wayfair app

There's no shadow underneath of the furniture, so it looks floating.

The furniture can be placed over an area where a door is supposed to be.

Define Problems

I also conducted user interviews to find out the users' pains and needs based on their experience with searching, purchasing, assembling and laying out furniture.

Because this is the most common pain point that comes up for users, this project focuses on creating instructions that make use of AR.

100% of the participants had bought flat-packed furniture; a bed frame, a drawer, a dresser, a shelf, etc. People buy flat-packed furniture not only from IKEA, a leader on flat-packed furniture store, but also Amazon, CB2, West Elm, and Target. 80% of the participants stated that assembling was stressful.

This concept is still hypothetical because I didn’t have the skills and the team to build an actual prototype in AR. However, I believe this idea could be a big help for Heem users.

users needs

1/1

Persona

Persona-02

Persona-01

Persona-02

1/2

How Can AR Help Users?

The main reason why participants said that assembling was stressful was because the instructions were not clear. IKEA is notorious for its no-word instructions, but participants also get confused and stressed even when the instructions were written out.

"Figuring out which tools are needed and which piece goes to where."

"I think I screwed the screws that held in the seat the wrong way.

As a result, the screws stuck through the seat and it was painful to sit down."

"I miss placed the top part and the bottom part because these parts looked the same."

Hypothetically speaking, computer vision could be helpful in helping users assemble their flat-packed furniture. Computer vision processes the position and orientation of each part of the furniture, and the holes and slits on some parts. AR system then overlays additional images on the parts. In that case, AR can direct users where to put screws, as well as check whether they placed the parts in the correct places.

Idea sketch

IKEA's famous no-words instruction gave me the idea that AR can make instructions clear without adding words. IKEA's no-words instructions make sense for the business side because they can save time and money for translation.

This shows how the app checks whether the parts place the correct direction or not. The users I have interviewed told me that some parts were so similar that they often installed them in the wrong places, so if they had a way to confirm which parts are correct and which direction some parts are supposed to face, it would be helpful.

Overall

This project was challenging for me since AR is new to me and many users. In this project, I focused on how AR can help users to solve their problems or provide additional values to their goals. Even though this is a hypothetical idea for now, this technology will be real in the near future.

Through this project, I believe that AR will provide tremendous opportunities for new user experiences. AR can drastically change shopping behaviors, visual designs for mobile and tablet, and more. I just can’t wait to see these new developments in AR, and hopefully, I am one of the designers who would make them happen!

AR brings users actual stores to see what exclusive coupons available.