Kristi DeSimone

Wayfair Service Pro

Wayfair Service Pro is an app that professionals use to find jobs assembling furniture for Wayfair customers. My team owns the customer experience, the pro experience, and the internal tools our operations team uses.

The problem: Pros were cancelling jobs at an average rate of 22%. Additionally, there was no up to date user research to understand the pain points of our pro users when I joined the team.

The objective: Gather qualitative user insights and generate solutions to decrease cancellation rate and improve pro experience.

The team: I worked closely with my product manager, engineering manager, full stack engineer, and iOS and Android engineers. I also consulted with our research operations team and pro management team.

Ladders

My Process

User Research

Conducted 10 user interviews to re-baseline our understanding of our pros

Share Insights & Hypothesis

Share my findings with a cross functional team and prioritize next steps to improve user experience

Design & Test

Design a solution to improve identified pain points, release as a test, and measure the impact

User Research

Gathering qualitative insights from our pro users allowed me to identify a correlation between their pain points and the high cancellation rate my team was looking to address.

Part 1: Survey

76% of Pros want to be doing more work with Wayfair
"The competition on the app is insane."
"Some of the jobs are outlandishly far."

Part 2: Interviews

"There's literally no time to look at a job and assess where it is, how far it is, how much it pays, you just have to click if you want it...there was one job that was an hour away and I had to cancel it unfortunately."

Insights & Hypothesis

Pros do not have adequate time to assess job details (location, duration, pay, etc.) and will cancel jobs that do not meet their requirements (too far, pay too low, etc.)

If we offer a job to a smaller group of pros, there will be reduced competition and more satisfied pros

If we offer a job to the closest pros, and make the distance from job more clear, pros will be less likely to cancel the job

Past: no job matching logic

Suggested: distance based matching

Design

After sharing my insights with my cross functional team and agreeing to move forward with a distance based matching test, I worked with a copy writer to create a new job notification to make it more clear the job being offered is closer than usual.

Our existing notifications:

All pros receive the same notification at the same time, regardless of their distance from the job.

New notification:

Send job notification to the closest pros first, then to a farther group if not claimed in X amount of time

Technical Constraints

Due to technical limitations, we could not simply implement our notifications to go to the 5 closest pros, or all pros within 5 miles of the job. Instead, we had to work within our existing proximity logic. This meant we had to separate all the pros in a market in to groups based on their proximity to the job.

Rank Pros

Pro 1: 1 mile away
Pro 2: 3 miles away
Pro 3: 5 miles away
Pro 4: 9 miles away
Pro 5: 14 miles away
Pro 6: 17 miles away
Pro 7: 22 miles away
Pro 8: 27 miles away
Pro 9: 32 miles away
Pro 10: 39 miles away

Group Pros

Group 1: Closest 0-15% of pros (avg. up to 6 miles away)
Receiving new notification
Group 2: Closest 15-50% of pros (avg. up to 6-14 miles away)
Receiving new notification
Group 3: Closest 50-100% of pros (avg. up to 14-60 miles away)
Receiving old notification

This would allow for a smaller and closer group of pros to be sent the job to claim first, reducing competition and increasing targeting of jobs. However, because of this, it would not be possible to message the actual distance from the job in our push notifications to pros since all pros in a group could be a different mileage away.

Test Results

This new strategy was implemented as a difference in difference test to compare 5 test markets to 5 similar control markets.

-7% Pro Cancel Rate in test group (+0.25% in control)
-18% Pros clicking already claimed notifications (-10% in control)
Qualitatively, the pros we spoke to did not notice a change in their recent job offers

My Thoughts

Though pros did not seem to notice the new distance based matching strategy in practice, the metrics we tracked indicate a success. We saw less jobs being cancelled and less pros clicking in to notifications that were already claimed. This will decrease the feelings of competition on our app, leading to happier pros and happier customers as their assemblies are not being cancelled!