Room Decorator - Decorate anytime anywhere

Role: Lead 3D Interaction Designer and Prototyper to launch Room Decorator as a new way to shop on Amazon

Duration: 9 months

Team: 3 designers, 1 PM, 10's of engineers, QA and business analyst

Testing and de-risking

I realized that the team was constrained on developer resources and decided to create a Unity prototype to validate two key assumptions:

1. Users will notice the “Tilt phone up” message.
2. Users will click on “Tap to place” button.

I conducted a remote usability test with 20 users and found that the majority of the users were able to understand the new interaction and successfully place the 3D model on their room floor. I decided to further refine the concept.

First A/B test

We launched an A/B test comparing the new design with the old design to a small percentage of customers. While the test was running, we saw a significant drop in the number of customers placing the product. A detailed analysis showed that some customers did not tilt their phone up, and others did not "tap to place" the 3D model on the floor.

Both of these hypothesis had been tested before, but I realized that the participants in the user test were disproportionately more adept at technology than Amazon customer base. Great learning for future, but I had to move quickly to improve the design and launch the feature.

Fast Iteration

We came up with 3 improvements after looking at metrics and conducting another usability test with the launched app.

1. I proposed making the “Tilt phone up” very prominent by showing it in the middle of the screen. We had observed that significant percentage of customers were not tilting their phone up.

2. I proposed to keep the product visible once the customer had tilted their phone up and not make it disappear if they tilt their phone down again.

3. We also noticed that customers were first seeing the 3D models very far away from their feet. I proposed to reduce the distance of the minimum zone so that the customers can see the product sooner when they are tilting their phone up.

We relaunched another A/B test with all the changes above.

We achieved a multi-million dollar increase in Amazon product sales.
We increased the average distance where the customer placed the 3D model.

We realized we had made too many changes at once in the release, that made it difficult to identify design decisions that moved the business results. As a result it became hard for us to iterate on the feature to improve the right metrics. We incorporated new processes to ship the minimum number of changes in a release and split releases if necessary.

I also realized that getting good results in a usability test on a prototype does not mean that every customer will be successful. This is especially the case with AR as customers are experiencing AR for the first time. I proposed to incorporate extra time for prototyping and iterating in our planning for future projects.

*Disclaimer: These are my personal views and do not represent the views of Amazon. The numbers are made up for confidentiality reasons.