Case Studies

Situation

When I took over E-Commerce at National Vision, we learned that some customers were not able to see their store orders in their online accounts when they signed in. Because we were selling prescription items like contacts and glasses, this is a huge problem. The average user doesn't know what contact lens they wear, and this creates friction in the purchase experience. It was causing headaches in operations as well with our CSRs needing to modify orders and field a ton of extra calls. Our store customers simply wanted to reorder the lenses they purchased in store, but we were making it difficult.

Task

We started by understanding the problem. This wasn't easy, but by working with our engineering team we were able to determine that in some user flows, the website was creating a new account for a user that would essentially lock them out of their store account. They would forever have a "store" account and an "online" account. I worked with our UX team and our business analyst to create detailed user journey flows and we added requirements in the flow charts. This made communicating the complex changes and flows we needed to engineering mush easier.

Action

We also met with operations to make sure that the changes we were contemplating would fix their issues as well. It was important to have them involved and get buy-in. The UX team produced detailed mocks in Figma and we connected these mocks to our flow charts and provided acceptance criteria to the engineering team. We were able to deliver the required changes in 3 Agile Scrum sprints. I conducted UAT along with UX and our BA, and the release was very successful. 

Result

Post-release we were able to produce reporting that showed we had stopped the creation of the duplicate accounts and greatly improved the user experience. Now that we had "stopped the bleeding" we needed to prioritize changes that would fix the existing user accounts with 2 different accounts.

Situation

We launched DiscountGlasses.com in 2014, selling prescription glasses direct to consumers. We had sourced some great budget brands and created 4 private label frame brands as well. We had an outstanding website, great products, and prices that would beat most retail stores. We were growing by about 30% per year, but we were seeing our competitors grow more aggressively. We wanted to scale the business, but we needed to do it in an efficient manner.  Our conversion rate was around 2%, so we knew that we would be able to better leverage our marketing spend if we were able to improve this rate. So why were users not converting?

Task

We set out to find out by using a mix of qualitative and quantitative research methods. We created three versions of a quantitative survey. One was for our existing customers and was geared towards understanding what these users liked about our experience and brand, and what made them convert. We also created a more general survey that we sent to a list of users who we identified as eyeglasses wearers. A third survey was sent to contact lens customers who had purchased on one of our brands, DiscountContacts.com. As part of the survey, we solicited users for phone interviews. I personally conducted about 20 phone interviews using a script that I created with the Marketing team.

The output of this research was very useful in identifying the user problems and motivations for buying glasses online. We found that users shopped online for glasses to save money and to find brands or styles that were not available in their local shops. The problem was that they were unsure if a frame would look, fit, and feel great, and whether it was poor quality because of the lower prices. This was after all a fashion and health item.

Actions

Armed with this research, the team got to work brainstorming possible solutions. Our list included Home Try-On, Virtual Try-On, Try Before You Buy (TBYB), User Generated Content and Reviews, and Virtual Opticians, among others. We conducted sessions for each of these solutions and outlined assumptions we needed to test, estimated the engineering cost to implement, as well as other operational costs, and scored the options using the RICE framework to determine viability, feasibility, usability, and value. RICE stands for Reach, Impact, Confidence, and Effort. For example, Home Try-On is a great solution for the user and we hypothesized that it would create the most user value, but it is super expensive to operationalize, and therefore not very viable. We knew from past tests that Virtual Try-On gets great user engagement, but it does not directly cause conversion increases. We thought that Try Before You Buy might provide value and be more viable than Home Try On, but we had some assumptions to test. The biggest one was would it improve conversion and deliver value to users.

We built an MVP by simply not charging users' cards for 14 days and designing content on the PLP and PDP explaining the benefits of Try Before You Buy. We believed that conversion would increase, but we also knew that returns could also increase. We limited TBYB to 2 frames and we included prescription lenses. This meant that we would potentially dispose of the lenses if users returned the glasses. We had a baseline return rate and we wanted to know how TBYB would impact return volume as well.

Result

We launched the feature using a feature flag so that we would be able to turn it off quickly without a deployment. We also only offered the feature to a small cohort so that we would eliminate any seasonality or other confounding factors. We ran the test for 4 weeks and then turned it off to gather data and put together the results. We were able to demonstrate a statistically significant conversion lift when we offered TBYB, but we needed to wait for 14 days to tally the returns impact. What we found was that returns did increase, and it was enough to offset the conversion gain, so the feature wasn't viable. 

In the end we implemented Virtual Try-On and we increased the prominence of User Generated Content like images and reviews. These features did improve conversion and helped us to deliver solid growth, but we continued to test alternative ways to solve the user problem. 

Coming soon...

Coming soon...

Coming soon...