Experiment Three: August 2020

Bottle Type, Ad Design, Badges
Marpipe

Does Creative Matter: a series of experiments in which we run ads targeting marketers, telling the marketers that these ads are part of an experiment. The goal is to prove how small creative changes within your ads can have a large impact on performance.

In August, we experimented with the bottle type of our fake product, the ad design, and the presence of badges in our creative. This experiment ran for 7 days, spending $250 and accumulating 2,697 clicks.

Outline

This experiment included the below variable groups and values.

  • Bottle Type: Original vs Dropper vs Small Bottle
  • Ad Design: Original vs Left Side vs Four Squares
  • Badge: Black&White vs Blue vs None

CTR Results

The most impactful creative component from this experiment was the Original Bottle, with an avg. CTR of 6.73%, with the Original Design right behind it.

Click Results

The Original Design was responsible for 14.6% of the clicks in this cycle. While the worst performing element by far was the Small Bottle with under 8% of the total clicks from this cycle.


After running three cycles and only spending $650, we've discovered a top performing ad variant with a CTR of 11.09%! We've challenged our winning components and they've proven that they are the best creative elements once again.

In the next cycle, we'll expand our experimental strategy to try and improve our winning elements further!

Start Testing on Marpipe Free

How to Run a Multivariate Test

The Beginner's Guide

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Tiffany Johnson Headshot

How to Run a Multivariate Test
The Beginner's Guide

Plus, Get our Weekly
Experimentation newsletter!

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
CompanyAbout UsPrivacy PolicyTerms of Use
Careers
Hiring