How UX research shapes good deliveries
As product designers, we are often told user research is too expensive, or that it takes too long to run it and bring back results. And I'd say that usually it is difficult to build arguments to fight for user research when a lot of companies demands high-quality deliveries in no time with little to no information about user needs. With experience (and time) you learn how to advocate for UX research, and one way of doing that is using other projects as successful references, and this project is (proudly) one of my arguments nowdays.
Wake Commerce is an e-commerce platform for enterprise-sized clients based in Brazil. When I started working with them I noticed that the engineering department cared a lot about keeping their product well-built, both due to security standards and future maintenance. There was also a need to redesign some features/microservices with a user centered approach.
So the challenge here was: how to design (or redesign) features that Wake clients use everyday, with close to no negative impact for the users, and also guaranteeing that maintenance would be easy and fast? Due to an NDA, I can't disclose all the details about the final product, or share pictures, but I can write about the process and some nice findings.
The project
I worked on dozens of projects for multiple features at Wake, but this one was the most complex project for sure. It was the promotion> microservice, which is used everyday and has a huge impact on the GMV of the clients, so, like, I really really couldn't mess it up. Promotions was one of the first ever implemented micro-services, so it was really old school looking, and also very confusing, with excessive navigation, insufficient information, and non intuitive options and configurations. From the beginning I made it clear that including clients in the process was not optional, but essential for delivering an effective solution.
Discovery
Due to high demands for other projects, I had to start the project with only one product analyst helping me. I drew a timeline for the project and together we started mapping the micro-service and collecting user data. After mapping and rating all the features - from the most to least used, most to least complained, the ones with tons of bounce rates, etc - we started to dig into all the support tickets, NPS rates and feedback emails, and talked to both sales and customer success departments. This helped us understand what our users want, so that we could ask them why.
With all the data, I made a script for interviews and my colleague selected and contacted some of our tier 1 and 2 clients. And then we had achieved our first milestone in this project: the clients were excited to talk to us!!
We asked them about every feature, the overall experience creating and editing promotions, the pain points, the highlights, and also heard tons of suggestions and good references from other platforms and softwares they used. With all of that noted in a spreadsheet, it was time to get my hands dirty: drawing time!!
Building the solution
Call me old fashioned, but drawing has always been my first step into ideation. There's something about pen and paper that Figma will never have.
During the user interviews we had a couple of comments along these lines: “I don't know which configurations or options are obligatory and which ones aren't”, “I don't know when I can close the page, is the promotion saved after the third step? Because it has a “Save” button, so is it saved or do I have to go though the following pages?”, “I never apply stamps to the promotions, so I don't need the last page” and, in contrast, this one: “I always use stamps, so it's annoying that I have to keep clicking “Next” until I find the stamps page”. Taking all those comments into consideration, I decided to build the “create a new promotion” form as one (long) page, with sections, instead of a 5 page wizard, so the user can have a clear notion of where they are in the process of creating a new promotion, and also, no infinite clicking until they find whatever feature they're looking for.
Before moving ahead I had to make sure this idea was feasible, so after taking a look at our design system's components I had a long conversation with devs and engineers so we could negotiate the interactions that my layout proposal would require.
During the next 2 months I would design high fidelity prototypes, weekly validate them with my design team, and then validate it again with the engineering team. Keeping everyone on the same page is how I like to work and in this case was key to the project success. Every now and then, during the validation meetings, the engineers would find some technical constraints, or I would need help figuring out the promotions calculations, and being close to them and able to discuss all those things as soon as I finished the prototypes made it possible to make changes with no impact on the deadline.
User testing
Prototypes finished and validated means time to bring the users back to the process. It took a couple weeks to build the interactive prototypes for the user tests, write the script, the survey, and also contact and schedule the tests with our clients/users.
I structured 8 tasks in total, some of them were as simple as “activate/inactivate a promotion”, others were more complex like “add a stamp to a promotion”, or “add a discount for customers from a specific region”. Everything ready for the tests, the last step before entering the Google Meet room was to set some metrics and expectations for the results. At this point in the project, a PO came to work with me and my colleague. She helped a lot with the metrics, especially because I was unsure about just taking the average numbers you find everywhere on the internet, but she was confident we could achieve a 4 out of 5 rate for the NPS and over 70% usability rate.
In addition to tracking key metrics, I established three core objectives for the usability test:
- Compare the intuitiveness of the new journey to the existing (the old 5 page wizard) workflow;
- Identify learning curve challenge;
- Understand the best ways to support users in the new journey - using guides and help documentation.
The good, the bad and the ugly
I'm just kidding, there's no bad or ugly here. To be honest, I did (and still do) believe in the solution I proposed. It checks every box I look to accomplish in a project, it's consistent to the product as a whole, it's clear, and every step has the information the user needs to finish their job, and it respects the design system as a system, with rules and hierarchy. But I got caught off guard with the results.
The tests got delayed due to our clients schedule, and I ended up not being able to attend all the meetings for the tests. I had a 2 week trip planned for months for my vacations, and could only attend the first round (week) of tests, where we did get good/average results. But I could not expect that, while I was eating frietjes, stroopwafel and lots of cheese, my colleagues were there in the office, seeing our test get amazing numbers.
Our users got a 99% success rate in the tasks proposed, 71% had no problem understanding the tasks, to 28% that had, and one of the best numbers: 100% of the users had a good impression of the solution proposed in the test (image below with some disclaimer about these numbers). Some additional metrics:
- Misclick rate: 8%;
- Usability score: 82.1/100;
- User score (NPS): 4.2/5.
This project was for sure the most challenging I've led, and it's now one of my proudest achievements in my career. The positive feedback after the release kinda surprised us; it wasn't just about the new layout, but about our clients feeling heard and seeing their feedback lead to improvements. Once again, it was amazing to see how we are able to create solutions that truly address the user and client needs when we bring them to the design process; how close to solving their problems you are when you not only listen to users but also translate their feedback into design decisions. Good design can't be done by yourself alone.