The human experience

building brand value through data-driven marketing

Since 1996, we've helped leading brands create interactive customer experiences that drive measurable business value.

The Value of A/B Testing

Invest in your investments to avoid getting burned.

By Noah Blum • Senior Project Manager, Antics Digital Marketing • April 11, 2017

The time to redesign one of your customer-facing marketing materials has come. Your website/newsletter template/landing page/etc. has gotten stale and you finally have time to address the issue. Keeping your materials fresh is paramount, but certain steps should always be taken before updating these assets and pushing them out to your audience. With some due diligence you can minimize risk, protecting yourself from unwanted results such as decreased performance and increased costs. The solution? Invest in your investments to avoid getting burned.

The Real World Example

For years we handled production for a newsletter that was sent to a large audience. Every month, the editor would send me the content, and we would hit the ground running with development and QA. One day we were informed that they’d redesigned the newsletter template – and the scheduled send was about 4 days away. Being curious, I asked about the impetus for the design change and was told by the editor that he wanted to switch to a template that was far more focused on imagery, having very little copy to describe each article linked to from the newsletter. He believed the copy was exhaustive and the focus on imagery would resonate with readers, thus leading to more clicks. As no research or testing was done to support this belief, I suggested we do some testing before taking the new template live to the full audience. Ultimately, though, that is not the direction we took.

We pushed forward and sent the newsletter to the full audience. The result? Average CTRs dropped immediately and open rates followed suit over the coming months. They’d spent time and money creating this new template and unfortunately the result was decreased performance. 

Fortunately there are a number of strategies that can be employed to maximize performance, both in regards to preliminary research and the rollout of a new template or design.

What steps should be taken before a redesign?

There are many ways to mitigate risk and maximize results when it comes to updating marketing materials, most notably research and analysis. First, conduct general market research. Investigate high-level best practices and trends specific to your asset to learn some general dos and don’ts. This will ensure you aren’t making any egregious errors.

In addition to general market research, be sure to engage in competitive research as well. Subscribe to your competitors’ newsletters, review their websites, their landing pages, their ads, etc. What is their brand look and feel? What design elements are they using? What content are they focusing on and how are they pushing it? What are the perceived strengths and weaknesses of their various materials? Know your competition, leverage what you think they’re doing well and fill the gaps in the places you think they could be doing better. By doing so you help position your firm as a leader.

Surveys are also a great tactic to employ. You have a mailing list, and everyone on it has an opinion. Send out a short survey designed to provide insight into what they like, what they don’t like and what they’d like to see more of. Analyze the results to help guide your decisions going forward.

Analyzing all of your existing data is imperative as well.  You can learn a great deal from your audience’s actions, so be sure to measure their behavior and adjust your strategies accordingly. Analyze open rates and click through rates, review page views, look at heat maps, analyze user flow throughout your site, measure engagement, review the performance of various calls to action. Such metrics let you know what people are gravitating towards so you can give them more of what they want.  Listen to them. The data may feel endless but pick a few key performance indicators to measure before and after your redesign.

Finally, if you use a marketing agency to help with your communications, it’s a good idea to consult with them before undertaking any such redesign. As a trusted partner with expertise in communications, they should be able to help with all of the tactics above, help you save money and maximize performance, and help you craft the right strategy and approach for a redesign.

The Rollout

Once you have a new template or design you like and you think will resonate with your audience, employ A/B testing and iterate accordingly. For newsletters, test two or three email layouts, all with the same subject line and content. Send template A to a third of your audience, template B to a third of your audience and the existing template to a third of your audience. From there, you can see what performs best and continue A/B testing until a final template has presented itself. Then test your subject line and calls to action. Such A/B testing can be applied to all of your digital marketing materials across the board, and you can apply multivariate techniques to test more than one variable at a time.

The Value of A/B Testing

While you may spend more time up front, employing the above strategy comes with many advantages, including:

  • Saving money in the long run by investing in the most engaging creative direction
  • Increasing the performance of your communications program
  • Knowing your decisions are based on the “feedback” of your audience
  • Gaining insights. As you do more A/B testing of different variables, you can start to gain a real understanding of what resonates with your audience, which can help you make more informed marketing decisions.

Now that you have your new, optimized template design, take what you’ve learned and put it into practice! Continue to look at the same metrics, comparing overall CTR, open rate and conversion rate, while also looking at detailed analytics so you know what type of content is resonating, how frequently your audience engages with your content, what imagery people respond to, and how they flow through your newsletter or pages.

Your audience’s tastes and needs will change with time and to succeed you’ll need to change with them. If you do, you’ll achieve long term engagement and customer retention. Deliver value and receive value in return. Invest in your investments.

Comments are closed.