Beta

Experiment 1: Product Experiment in June 2020

Get an in-depth look at product experiment. Discover how even the smallest changes can make a big difference in your ad performance with Marpipe.
Marpipe

Does Creative Matter: a series of experiments in which we run ads targeting marketers, telling the marketers that these ads are part of an experiment. The goal is to prove how small creative changes within your ads can have a large impact on performance.

Most advertisers agree that high-quality product shots can lead to incredible performance lifts across campaigns. However, many of them fall short while trying to find the best context for their products. Does a flat-lay display of the product work better or a front-face display? Should the background look real or just be a single color? How will ingredients impact my ad performance? These questions are all very important and the answers can change the performance the of the assets you create by incredible amounts.

On June 3, 2020 we launched our first Does Creative Matter experiment of the year. This click-optimized experiment ran from 6/3 - 6/9, spending $150 and accumulating 1,493 clicks.

Outline

This experiment included the below variable groups and values.

  • Product: Bottle vs Can
  • ProductShot: Flat-lay vs Front-face
  • Background: PurpleBG vs WoodTable
  • Ingredients: True vs False

CTR Results

The clearest CTR outlier in the entire experiment was the Bottle Product, beating not just the Can, but all other creative values. The WoodTable and Flat-lay values were the winners in their respective groups, and not including Ingredients was more effective than including them.

Click Results

The Bottle Product collected over 14% of the total clicks in this experiment (861 clicks), while the ads featuring the Can Product had the lowest amount of clicks.


With the findings we've collected here, we'll dive deeper into our winning values and challenge them in July. Stay tuned!

Boost ad performance in days with a 7 day free trial.
Claim Trial

How to Run a Multivariate Test

The Beginner's Guide

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Tiffany Johnson Headshot

How to Run a Multivariate Test
The Beginner's Guide

Plus, Get our Weekly
Experimentation newsletter!

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Outliers is the weekly newsletter that over 10,000 marketers rely on to get new data and tactics about creative testing.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Are you crazy...

about catalog ads? You’re not alone. Join over 8,000 other marketers in The Catalog Cult - the world’s best newsletter about catalog ads.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.