How to send A/B Split Test Emails
Posted by Dave Zulu, Last modified by Olivia Tang on 01 May 2020 03:31 AM
To get the best engagement possible sometimes it is best to test your recipient reactions to certain campaign related initiatives. For example, open rates based on variable subject lines or which buttons are receiving more clicks.
A/B split testing is especially useful when trying to decide on using condition based content versus short and sharp eDM based strategies.
Split testing refers to the recipients being selected. Splitting the sender group (randomly or not it is open for debate) into at least 2 test groups each receiving just one email that contains the content being tested is the simplest way to start split testing.
Here we are going to use the Zulu eDM Index sending with personalised subject lines and the comparative reports after the send.
Test: Does using personalised subject lines improve open rates?
Total Number of Recipients (subscribed): 1000
Test Number Recipients (subscribed): 100 per email, 200 or 20% total
1.) Template is ready to be added to a campaign
2.) The total number of SUBSCRIBED recipients is known for the target group(s).
3.) Both campaigns are ready to have Campaign Information added
1.) For each campaign go into the campaign information and enter the required subject line. The screen shots below show an example of a generic subject line and the second campaign uses a more personalised approach.
Upgrade to the Subscriber Dashboard is now live!
##_people_fname#, Your Subscriber Dashboard Upgrade is live!
2.) Set your indexing to split the send list.
At the bottom of the Campaign information screen you will see Set Partial List Send Limit? Check that box and for the first send enter in the database starting index (from 0 and up) and the number of contacts you wish to send to, in this case 100.
Repeat the process for the second campaign but please note, the system will look for the first 100 contacts that are SUBSCRIBED. So if the index started at 15 then it is quite possible that if 20% of your database is unsubscribed then the the indexing would have finished at contact 135. So we recommend to go to give plenty of room for error. so in this case we are starting the indexing at 250.
3.) To perform a true split test schedule both campaigns at the same time using the scheduler. Please remember these servers are set to GMT +8.
Once your campaigns have been sent we recommend waiting just 24 hours. That way your preferred campaign to the remaining proportion of your recipients. To avoid duplicates being sent to the split testing recipients please remember that for each split email that was sent to "Add Contacts That Opened The Email" to a new group and add that group to the block list for the final email being sent.
Open the campaign stats for either campaign that was sent and click on the A/B results tab. Select the other subject line and copare the results: