Back

Experiment Three: August 2020

Bottle Type, Ad Design, Badges
Marpipe

Does Creative Matter: a series of experiments in which we run ads targeting marketers, telling the marketers that these ads are part of an experiment. The goal is to prove how small creative changes within your ads can have a large impact on performance.

In August, we experimented with the bottle type of our fake product, the ad design, and the presence of badges in our creative. This experiment ran for 7 days, spending $250 and accumulating 2,697 clicks.

Outline

This experiment included the below variable groups and values.

  • Bottle Type: Original vs Dropper vs Small Bottle
  • Ad Design: Original vs Left Side vs Four Squares
  • Badge: Black&White vs Blue vs None

CTR Results

The most impactful creative component from this experiment was the Original Bottle, with an avg. CTR of 6.73%, with the Original Design right behind it.

Click Results

The Original Design was responsible for 14.6% of the clicks in this cycle. While the worst performing element by far was the Small Bottle with under 8% of the total clicks from this cycle.

After running three cycles and only spending $650, we've discovered a top performing ad variant with a CTR of 11.09%! We've challenged our winning components and they've proven that they are the best creative elements once again.

In the next cycle, we'll expand our experimental strategy to try and improve our winning elements further!

Read our full creative testing guide

SIGN UP FOR THE EXPERIMENTAL METHOD NEWSLETTER
TO GET THE GUIDE

Download Now
View Sources

Heading 1

Heading 2

Heading 3

Heading 4

Heading 5
Heading 6
"This is a block quote"
  1. This
  2. is
  3. an
  4. ordered
  5. list

This text is bold

The rich text element allows you to create and format headings, paragraphs, blockquotes, images, and video all in one place instead of having to add and format them individually. Just double-click and easily create content.

A rich text element can be used with static or dynamic content. For static content, just drop it into any page and begin editing. For dynamic content, add a rich text field to any collection and then connect a rich text element to that field in the settings panel. Voila!

How to Run a Multivariate Test

The Beginner's Guide

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Tiffany Johnson Headshot

How to Run a Multivariate Test
The Beginner's Guide

Plus, Get our Weekly
Experimentation newsletter!

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.