Experiment 1: Product Experiment in June 2020

Does Creative Matter: a series of experiments in which we run ads targeting marketers, telling the marketers that these ads are part of an experiment. The goal is to prove how small creative changes within your ads can have a large impact on performance.

Most advertisers agree that high-quality product shots can lead to incredible performance lifts across campaigns. However, many of them fall short while trying to find the best context for their products. Does a flat-lay display of the product work better or a front-face display? Should the background look real or just be a single color? How will ingredients impact my ad performance? These questions are all very important and the answers can change the performance the of the assets you create by incredible amounts.

On June 3, 2020 we launched our first Does Creative Matter experiment of the year. This click-optimized experiment ran from 6/3 - 6/9, spending $150 and accumulating 1,493 clicks.

‍

Outline

This experiment included the below variable groups and values. ‍

• Product: Bottle vs Can
• ‍ProductShot: Flat-lay vs Front-face‍
• Background: PurpleBG vs WoodTable‍
• Ingredients: True vs False

‍

CTR Results

The clearest CTR outlier in the entire experiment was the Bottle Product, beating not just the Can, but all other creative values. The WoodTable and Flat-lay values were the winners in their respective groups, and not including Ingredients was more effective than including them.

‍

Click Results

The Bottle Product collected over 14% of the total clicks in this experiment (861 clicks), while the ads featuring the Can Product had the lowest amount of clicks.

With the findings we've collected here, we'll dive deeper into our winning values and challenge them in July. Stay tuned!