Email List A/B Split Testing

Posted by Bryan in Email Marketing

When was the last time you used any type of testing in your email campaigns? Do you test the subject lines? How about segmenting your lists? What about testing timing? According to emarketer.com, between May and June 2009, 37.1% of marketers do not test their email campaigns.  

How about starting with a simple A/B test.  Here are a few quick steps:

  1. Take your list and define a small sample size that you believe will be statistically representative. Ten percent is a good number if you cannot decide.
  2. Grab two samples of your defined size. Label one “A” and the other as “B.”
  3. Define your parameters and how you will measure the response:
    1. the subject line,
    2. a design layout change,
    3. timing of the message,
    4. including live chat or no chat,
    5. link placement or quantity in the email body, etc.
  4. Setup two email campaigns.  Send the first message to List “A” and the second message to List “B” with the parameter changes.
  5. Run an analysis to determine which list had the best response.
  6. Send final message with the parameters that worked best to the remainder of your list.

Pretty straight forward, huh?

To help you out, here is a Python script for performing A/B List splitting.   Download the source code.  This script will create two random samples from your list based on the 10 percent (or your own) sample size.  You can use the list outputs to run your A/B tests all day long.   Enjoy!