Helping Etsy better understand their sellers

Background

In July 2020, Etsy requested that we bid on some CX/UX research they needed. We were awarded the work, and started in August 2020.

What they had

An overworked research team who rarely had time to investigate the tasks, needs, or realities of being an Etsy seller. They also had an in-house recruitment expert, who found us nearly everybody we met.

What they wanted

To better understand sellers’ tasks and end-to-end processes when listing an item on Etsy.

Project planning

The project was planned as 5 months for two Lead Researchers, Debbie Levitt and Larry Marine. Part of the reason for the length was that Etsy wanted a full report and presentation after each of the five groups.

We were completely on time for the entire project until the end. We needed an extra week for our final report, especially to select and generate our video clip montages, and were granted it. We stayed on budget for the project.

What we did

Goals, Planning, Strategy

This project was aimed at learning more about how Etsy sellers interact with a certain page/flow on the seller side of the site. Etsy had 6 target groups of sellers. We suggested 8-12 from each group for a 1-hour remote observational study with contextual inquiry.

One “fun” element that we added was a group that we would recruit from an outside panel who would not know we were working for Etsy. We recruiting for any online seller of handmade items, someone who matched the target Etsy seller. We watched people on the same page/flow on a variety of websites to analyze their task and learn what worked/didn’t work on both Etsy’s and non-Etsy sites.

Protocol, Script, Guide

We wrote a session protocol based on having 60 minutes with sellers from North America and Europe. All sessions were in English.

Recruiting, Scheduling, Communication

Most of the participants came from Etsy’s internal recruiter. We found 10 additional people who sell their crafts online, but wouldn’t know this study was for Etsy.

We used an online calendar system to schedule participants. Etsy handled incentives.

Session Execution

71 participants showed up. All were moderated and recorded on Zoom.

Insights, Opportunities, Suggestions

Due to the confidential nature of this project, we can’t share any of our insights, opportunities, or suggestions.

Deliverables

  • Task analysis (as is state) and optimized task flow (desired future state).
  • Etsy did not request personas, but as there were “typologies” that jumped out at us, we submitted those as well. These were four behavioral types where every participant we met fell into one of the four.
  • Individual reports and presentations for the first 5 groups. These reports were 15-40 pages long. Presentations were done live with Q&A time for Etsy staff and stakeholders from various domains.
  • Our final report, separate from our five earlier reports, was approximately 80 pages long. Our goal was to provide as many actionable insights and suggestions as possible.
  • Spreadsheet of all the 130+ main issues we found, their domain, and our scoring on their severity and potential priority.
  • Etsy had also requested any early concept sketches for any ideas we wanted to represent visually, provided we were clear that these were early ideas and not mandates by any means. We created two medium-fidelity Axure prototypes showing early concepts on potential improvements to two areas of Etsy’s desktop site.

Part of the reason our reports were so long and our documents plentiful was because our participants were so excited to be included that we heard many concerns, pain points, and issues that extended beyond the original scope of the project. Participants felt heard and cared about, and told us many stories that fell inside and outside of the original scope. We included these in our reporting and added any actionable suggestions we had in these areas as well.

Our contacts at Etsy were planning to slice the report and documents by domain, and distribute the information and suggestions to each team that owns an area we researched.

In the last project meeting, Debbie said, “I feel like we’ve given you two years of work.” Our main Etsy contact said,“You’ve given us five years of work.”

Additional research work

Etsy was so happy with our work that they asked me to propose a follow-up card sort related to the same area of the Etsy site that we had just researched. We ran an unmoderated card sort, which received over 50 responses, and the identical card sort as moderated research sessions with 12 participants. All were recruited by Etsy.

Etsy then received a report on the findings from the card sorts, actionable suggestions, and another medium fidelity Axure prototype showing an early concept for a desktop page.

This research project was completed in roughly one month, December 2020.

Happy client

Before quitting Etsy during our project, our main contact there was well-known information architect and author, Abby Covert. She later left Debbie a glowing recommendation on LinkedIn:

“I hired Debbie and her team to conduct some critical observational research (at scale!) for my team during my time at Etsy. Debbie was such a treat to work with and brought so much value so quickly. Her observational research skills are stellar and her ability to synthesize what she sees into the take aways that a team really needs to understand is such a breath of fresh air in a sea of long research reports and presentations. Don’t hesitate to hire Debbie or DeltaCX for your next project. You can thank me later.“