Experiment One: June 2020

Product Experiment

Does Creative Matter: a series of experiments in which we run ads targeting marketers, telling the marketers that these ads are part of an experiment. The goal is to prove how small creative changes within your ads can have a large impact on performance.

Most advertisers agree that high-quality product shots can lead to incredible performance lifts across campaigns. However, many of them fall short while trying to find the best context for their products. Does a flat-lay display of the product work better or a front-face display? Should the background look real or just be a single color? How will ingredients impact my ad performance? These questions are all very important and the answers can change the performance the of the assets you create by incredible amounts.

On June 3, 2020 we launched our first Does Creative Matter experiment of the year. This click-optimized experiment ran from 6/3 - 6/9, spending $150 and accumulating 1,493 clicks.


This experiment included the below variable groups and values.

  • Product: Bottle vs Can
  • ProductShot: Flat-lay vs Front-face
  • Background: PurpleBG vs WoodTable
  • Ingredients: True vs False

CTR Results

The clearest CTR outlier in the entire experiment was the Bottle Product, beating not just the Can, but all other creative values. The WoodTable and Flat-lay values were the winners in their respective groups, and not including Ingredients was more effective than including them.

Click Results

The Bottle Product collected over 14% of the total clicks in this experiment (861 clicks), while the ads featuring the Can Product had the lowest amount of clicks.

With the findings we've collected here, we'll dive deeper into our winning values and challenge them in July. Stay tuned!

Read our full creative testing guide


Download Now
View Sources

Heading 1

Heading 2

Heading 3

Heading 4

Heading 5
Heading 6
"This is a block quote"
  1. This
  2. is
  3. an
  4. ordered
  5. list

This text is bold

The rich text element allows you to create and format headings, paragraphs, blockquotes, images, and video all in one place instead of having to add and format them individually. Just double-click and easily create content.

A rich text element can be used with static or dynamic content. For static content, just drop it into any page and begin editing. For dynamic content, add a rich text field to any collection and then connect a rich text element to that field in the settings panel. Voila!

How to Run a Multivariate Test

The Beginner's Guide

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Tiffany Johnson Headshot

How to Run a Multivariate Test
The Beginner's Guide

Plus, Get our Weekly
Experimentation newsletter!

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.