A/B testing is effective in determining the best way to communicate with your customers.
Here are some examples of what A/B testing may be used for:
- Comparing subject lines
- Promotions or informative content—what do your customers prefer?
- Discounts—monetary vs. percentage
- Embedded links in buttons or in text
- Product images vs. scenic images
The main purpose of A/B testing is to determine the impact of a small difference in a message. If the differences are too significant it'll be difficult to draw any clear conclusions from the results.
Therefore, a good place to start is by testing different subject lines.
- Create a new email and tick the box A/B testing.
An A and a B version are created and you're able to enter different subject lines.
- When you've entered your subject lines and preheaders you see a preview for each version.
A/B testing cannot be used in combination with conditional subject lines and preheaders.
You may change the percentages allocated to the A and B versions of the message by dragging the black separator. However, note that the percentage allocated to version B will always be equal to version A.
There are several options to choose from when selecting how the final, or best, version should be evaluated. Please be aware that there are a few things worth considering before making this selection:
- It takes time for all the recipients to open the message.
- There may be some lag in the time taken for the recipient's email server to send a response to Engage. Keep in mind that it is desirable to have as much data as possible when determining the final version.
- If, for instance, your Engage setup lacks transactional data, there will be no point in using the options based on purchase sum.
- If the option used is based on purchase sum, please consider the fact that it may take time before the related transactions have been imported to Engage.
When you're in the design step of the message creation, you may choose that each respective module is only presented in one version, or both. You find this setting in the module menu on the right-hand side of the editor.
In the 'Send' step you're able to see a preview of the subject line and pre-header for each version.
Both test versions will be sent at the scheduled time to the defined percentage of recipients. The final and best version will then be sent to the remaining audience at the time delay you have specified.
If you choose to send batch-wise, only the winning version that is sent to the remaining audience will be batched up, not the test versions.
To review the results, locate the statistics overview for the specific email and choose which version you want to review in the top right corner.