Conducting successful tests

It's very easy to perform tests in Engage, but before you get started you need to define what to test and why.

Here is a guide on how to get started:

  1. Preparations: have a meeting to discuss the upcoming test and what kind of results you'd like to see from it.
  2. Form a hypothesis: Create a plan to define why you're performing the test and what you're going to test to achieve optimal results.
  3. Decide which channel to use: Will you perform the test using email or SMS message?
  4. Define the target group: Who are you going to send the test to? You can either create an A/B test which will have two test groups, or create a test scenario with several groups. Make sure that the contacts are randomly divided into test groups before executing the test.
  5. Test group sizes: The size of the group will vary based on the number of recipients in your defined target group. To ensure statistical significance your sample sizes shouldn’t be too small. There are scientific methods to calculate the size of the test group, but we advise using at least 5,000 recipients.
  6. The basic testing rule: Create your different versions but remember to only test one variable at a time.
  7. Analyze the results: Wait a defined amount of time to get complete results and start analyzing what worked and what did not.
  8. Adjust and implement changes: Based on your results, make changes for the future and prepare to test something else next.

Inspiration – what to test

The are many opportunities for testing, and we recommend that you create and follow a test plan to ensure that your tests will be carried out, but also to make the correct priorities. Remember that tests can be conducted in different channels, not just in email marketing.

Subject lines

How can you affect opening rates just by being creative with words? The subject line is probably the most commonly tested element of emails, and there’s so much to explore. Examples of what to test are:

  • Personalization – Does it matter if you address the receiver by their first name? Add a first name to one of the versions, let the rest be equal to see if the one using a name gets a better opening rate
  • Length – Do your customers prefer short and snappy or long and explanatory subject lines?
  • Topic – Test the effect of words like “free”, “included” or write discount in % or amount in your currency
  • Tone – Test the difference between a friendly or more direct expression
  • Urgency – What happens if you add a sense of urgency vs. not?

Examples of good subject lines:

  • Anna, earn double points today only
  • You're invited: 20 % off for members this weekend
  • 3 days only, 10 % off and free shipping!

Find out how important the preheader actually is.

  • Create one mail with and one without to understand the importance of supplementing the subject line with a proper preheader
  • Test different topics in the preheader to see which one your customers respond to


  • Save up to 50 % off this weekend during our summer sale
  • Exclusive preview of our fall collection inside
  • Thanks for signing up, here's your 15 % off voucher!

Send the same message, but in different channels to find out what to say where. Example:

  • Send an offer in both email and SMS to see which leads to more purchases.

Find out the most attractive offers for your different segments. Also test the effect of expressing discounts in different ways. Examples:

  • What works best in a welcome email: 3 for 2, 10 % off or Free delivery?
  • Does a sale work better or a voucher incentive?

Are reminders a tool for increased conversion rate? And which channels shall be used for optimizing the effect? Test the combination of:

Email + Email

Email + Text message

Email + Postal


Understand the importance of appealing images, creative headings or short vs. long body copy.


  • Do you customers appreciate inspiring content such as blogs or a short call to action?
  • Test different kinds of images, do people click more on products featured alone or on models?

To which extent do different call-to-actions affect click and conversion rates?


  • Should you write the link as text or have a button?
  • Does the color of the button make a difference?
  • Is it better to write ”View the product” or ”I really need new shoes”?
Day and time

Test day and time of day to optimize clicks and conversions.

There’s of course more to test than what’s mentioned above, but you need to start somewhere and continuously grow with your test strategy.

Was this article helpful?
0 out of 2 found this helpful



Article is closed for comments.