A/B copy experiments

A snapshot of routine copy experiments I ran at Booking.com

Signing in is worth it, promise

Problem: We needed to get more people to sign in so we could better understand their situations when offering them customer support. If users were logged in, we could proactively leverage information about their pre-existing bookings to help them better and faster. We just had to convince them of that.

Additional context: There was a route to customer support that users could take without logging into an account but it was a more convoluted (read: infuriating) experience. This was by design - the business wanted more people using their accounts in order to be able to offer better booking and post-booking experiences across the board.

Process: Using research insights and a little common sense to understand what was most important to people in this moment was paramount. They were looking for help, frustrated and frequently anxious about expensive travel plans. We needed to show users that we understood what they needed and clearly show them how to get it: by signing in.

Hypothesis: We’ll know this copy has been successful when we see more people logging in and less people using the “Continue without an account” option. When this happens, we can assume they better understand the benefits and purpose of logging in. Ideally, we would also see this impact reflected in behavioural metrics to tell us that, once logged in, more people were able to successfully navigate to the relevant customer support.

Result: We saw a significant increase in the number of people logging in and creating accounts. We also saw more people successfully navigating the help centre once they were there.

Cancellations made easy

Problem: The information about cancellations was vague and unhelpful, and the old CTA was both scary and untrue. Thanks to this daunting copy situation, most people who really needed to cancel a booking were not following through with the automated cancellation process. Instead, they were contacting customer service where real live humans could provide a more reassuring experience. All of this came at a needless cost to both the business and the customer service agents who could have better spent their time on more complex cases.

Process: Simply put, the goal was to make it make sense. We needed to reassure customers and make sure they knew exactly what to expect from this cancellation process. The complexity of cancellations also needed to be accounted for. With several different policies and potential outcomes, not to mention the general sensitivity required when talking about anything related to people's money, called for a more helpful, suggestive tone, accurate language and an easily understood call-to-action.

Hypothesis: We will know the copy works when we see more people successful completing the cancellation process combined with a reduction in customer service contacts about cancellations.

Result: Cancellations significantly increased while the number of people contacting Customer Service about cancellations dropped. Honestly though, I think the spacing is what really drove this one home. 😅

Consistent customer service entry points

Problem:  Due to a combination of legacy content and a large number of different product teams working on overlapping topics, the entry points for Booking.com’s customer help centre across the website and apps were all wildly inconsistent. This was confusing users, making it difficult for people to find help-related tools and information or to contact customer service. All of this was negatively impacting cancellation rates and causing unnecessary customer service contacts. 

Additional user context: Most people don’t use Booking.com on daily or weekly basis. So, if they’re familiar with one way to find customer support, they’re going to be confused when they come back in 6 months’ time and not be able to see the entry point copy they’re looking for. Additionally, thousands of people need Booking’s customer support every day. Most of those people have already exhausted all other options before calling or messaging, so contacting the company for help is typically the main priority for most users who reach the help centre.

We had to take this into account when changing something so integral to the overall customer experience.

Technical constraints worth mentioning: Creating a consistent baseline for entry point copy across all platforms required several different A/B copy experiments with varying base copy. I haven't included all of those here because at the time, desktop/web had the highest traffic and thus provided the best snapshot of how this copy was working (or not). 

Hypothesis: If we make customer support easier to find, we expect to see lower cancellation rates as more people will be able to find the help they need. 

Attempt one:

"Help Centre"

This tanked because the majority of users didn’t recognise what they were looking for in this navigation title. There was no mention of “Customer Service” specifically so it didn’t register for most people. We were also revamping the whole help centre at this time and the language from that bigger project had taken over my brain to the point where I thought the simplicity of this would stick, not to mention the way it aligns with many other companies’ help centres. Alas.

Attempt two:

"Customer Service"

This version worked better - more people were reaching the help centre in general - but it didn’t have quite the impact on cancellations and contact rates as I’d hoped.

Attempt three:

"Contact Customer Service"

Prompting users to proactively contact Booking finally did the trick. Cancellations decreased significantly at the manageable price of a thankfully “acceptable” increase in customer service contact rates.