Flight School Friday: 5 A/B Tests to Get You Started

Without analyzing opens and clicks, it’s hard to know what truly appeals to your customer. Surveys and focus groups have their limits since people often change how they act when observed – a phenomenon called the observer effect. The beauty of A/B testing is that it cuts out any guesswork, leaving you with actionable, data-driven insights. You can concretely determine which ideas should be used or discarded to build an ultimately winning campaign.

Here are five basic elements which are a good starting point for those new to A/B testing, and also fresh inspiration for the more experienced tester.

In the inbox, first impressions can go a long way. You may have crafted a SPAM filter-friendly subject line but what will convince the user to open your email over a competitor’s? A good subject line is informative, concise and intriguing. A/B testing allows you to compare open rates and find lines which drive engagement.

Emoji vs text-only: Emojis tie in the lighthearted, more personal nature of social media platforms (e.g. Twitter, Whatsapp). Depending on your contact list demographics such as age and social media usage, it’s worth testing if emoji usage drives higher opens. Do remember to  check browser and client support since they are still not widely adopted across devices.

Statement vs question: Asking a rhetorical question can persuade without being too pushy. Some consumers respond to  a more direct approach and others may feel a question drives a greater sense of urgency or personal connection.

Your sender name has a great impact on perceptions of your brand and on the nature of the relationship with your customer. Do you want to seem approachable and focused on the individual? Or are you a trusted authority on a topic? To find out the level of intimacy your customer best responds to, here are some useful A/B tests to run:

Company name vs individual: The first may cater towards users who want to belong to a community, or for a product that is more professional – while the second builds more of a personal customer-brand relationship.

First name only vs full name: This will help you gauge the level of formality your customers respond best to. For example, we would test “Sasha from Mailjet” against “Sasha Seddon from Mailjet”.

Different types of name: The name of a person can greatly affect how they are perceived. Try out different names to see which your consumer prefers.

An eye-catching design and persuasive copy aren’t the only items on the checklist for a  compelling call-to-action (CTA). It should work well with the overall layout while ultimately remaining the focus of the reader’s attention.   

Position next to image or text: Which other piece of content best complements and enhances the CTA, without detracting from it or cluttering the email?

Left, right or centered: If you don’t need to provide much information, a centered button might capture attention best. Or, positioning the CTA to the left or the right might work better with directional cues present in the email, e.g. arrows, or with the natural reading direction of the user.

Gut instinct plays a major role in decision-making. A recent study suggests that people make instant purchase decisions with their sub-conscious. How information is framed greatly influences customer response.

Percentage vs dollar off: Even if they will save the same amount of money, a percentage reduction may appeal more as there is no need to provide context. The reader can make a value judgement more quickly.

Day vs. Week: For service based products, try testing a day vs. week trial, such as 14 days vs. two weeks. Our hypothesis is that two weeks sounds like the better offer, but who knows? Some customers may feel like that’s too long of a trial. Testing will often uncover unexpected insights.

You have optimized your deliverability, content and design, but what if your timing is off? Here are some A/B tests to run to find out when your customers are most receptive.

Morning vs evening: Customers are more likely to read your email when they are less distracted or need to kill some time – this could be on the morning commute or when the working day is over.

At the start vs end of the week: Are customers most likely to be receptive after a refreshing weekend or during the week? Perhaps they completely unplug during the weekend and are most active on Monday mornings. Or, you may find that they don’t have time to check personal email during the week and are in the mood to purchase or engage during the weekend.

The insights gleaned from these email tests can also be carried over and integrated into your overall marketing strategy.

Hopefully these A/B tests have given you some food for thought. A/B testing is an incredibly powerful tool; there are a never-ending variety of tests you can run and with each one, you’ll understand your customers a little bit better.   

Have any useful tips or ideas you’d like to share with other A/B testers? Drop us a comment below!