Email Campaign A/B Testing: Examples, Ideas & Best Practices

Pat Walls

•

Updated: October 8th, 2022

What Is Email A/B Testing?

Email Campaign A/B testing involves sending two or more variations in your email marketing campaign to determine which version attains a higher open rate or conversion rate compared to the others.

In email A/B testing, you may want to compare how different email subject lines, CTA button color or images perform in terms of attracting conversions.

Ideally, A/B testing in the context of email campaigns involves a 50/50 split meaning half of your audience will receive variation A while the remaining half gets variation B.

Email A/B Testing Key Takeaways

Better email open rates and click-through rates, result in more website visitors and increased sales.
An A/B test eliminates decisions based on opinions, leading to a clear direction for your business.
A/B tests improve conversion rate and help you understand what messages resonate most with your audience.
With proper A/B testing, any business owner can attain a consistent path for growth.

One way to attain this is through running A/B tests on your email campaigns.

Understanding How Email A/B Testing Works

Here is an illustration of how the email campaign A/B testing would look like:

The ultimate goal for any email A/B test is to find out which variation of the email campaign garners the best results.

In our illustration above, Subject line A attained a 19.4% open rate, while Subject line B attained a 27.8% open rate.

Once the A/B testing is concluded and the winning version found, you automatically switch to the winning version. in our case, subject line B is the winner.

Therefore, our subsequent emails for this particular campaign will have subject line B.

Your email campaign A/B test might include the following simple steps.

Step1. Pick what you will test. For instance, in our case, we are testing the subject line.

Step2. Determine which metrics will guide you in choosing the winning version. [could be open rate or CTR]

Step3. Decide who you send the campaigns to / set the sample size [send to 25% of email subscribers]

Step4. Measure results and pick the best version.

Simple A/B Testing Strategy Vs Advanced A/B Testing Strategy

Email A/B testing can vary in complexity.

Simple A/B testing focuses on sending multiple subject lines, changing CTA color, or different images to test which one generates more opens.

Advanced A/B testing includes using completely different email templates to see which one generates more CTR.

If you are using emailing tools with advanced features, A/B testing your campaigns is very easy as you can use the email builder to create 2 different variations of your email campaigns.

What You Should Test In Your Campaigns

If you are using the best email marketing tools, you can test any aspect of your email marketing campaigns and attain improved results.

Here are few ideas of what to test in your email campaigns.

Call to action [Example: “Shop Now”! vs. “Shop our Spring Collection”]
Subject line [Example: "Enjoy 25% discount on our Spring Collection" vs. "Spring Collection Offers"]
Testimonials [include (specific testimonials) or Not to include at all ]
Email content layout [single column vs. two columns / varying placement of your email elements]
Personalization [Example: "Dear Mr. Davis" vs. "Our esteemed customer"]

Each of these variations is likely to have an effect on different aspects of the conversion process.

Email Campaign AB Testing Best Practices

Always test simultaneously to ensure time-based factors do not affect your results.
For more accurate results, test a larger sample as you can.
For quicker and easier Email campaign A/B testing, make use of tools available to you.
Testing early and often helps to attain the best results

Real-World Examples Of Successful Email A/B Testing

Here are two real-world examples of successful AB tests:

1. How Blenders Eyewear increased revenue by 48%

Blenders Eyewear, launched an email A/B click test campaign.

The campaign featured two versions of emails with different visual styles but similar images and calls to action. Version A got a 33% increase in clicks and a +48% lift in revenue.

2. How Designhill increased clickthrough rate by 5.84%

Designhill, one of the fastest crowdsourcing platforms did an email blast aiming at promoting its content and increasing the CTR.

Just mentioning the title of the blog in the subject line of the email got a majority of click-through compared to the version requesting recipients to review the post with the blog's title.

The company scored a 5.84% higher CTR and a 2.57% higher open rate by including the title of the blog in the subject line.

How To Run An Email A/B Testing Campaign

Know what you are testing
Focus on the frequently sent emails
Split the subscribers list randomly
Test highly divergent variants. (e.g, plain text versus HTML emails)
Keep in two variants
Wait for up to a week to test
Check if the results are statistically significant
Act on the results

Email A/B testing does not have to a strenuous or expensive exercise. Different emailing tools come with an A/B testing feature. Therefore, you will want to go for these advanced emailing tools:

Klaviyo

Klaviyo’s the ultimate ecommerce marketing platform trusted by 50,000+ brands, like Chubbies, Living Proof, and Hint. Deliver higher-value, personalized marketing experiences across email, mobile, and web.

Up to 250 contacts: Free!
Up to 1,000 subscribers: $30 per month
Up to 2,500 subscribers: $60 per month
Up to 5,000 subscribers: $100 per month
Up to 10,000 subscribers: $150 per month
Up to 20,000 subscribers: $375 per month
Pricing: Freemium
Twitter: @klaviyo
Website: klaviyo.com

Businesses using Klaviyo: