Experiment 5 (November 2020): Format, Tags, & Product Ads

Does Creative Matter: a series of experiments in which we run ads targeting marketers, telling the marketers that these ads are part of an experiment. The goal is to prove how small creative changes within your ads can have a large impact on performance.

In our November 2020 experiment, we wanted to challenge one of our ongoing winners (the Bottle) while also experimenting with new creative elements! This cycle ran for 7 days and collected 1,451 clicks.

‍

Outline

This experiment included the below variable groups and values. ‍

• Format: Video vs Image
• Product Bundles: Bottle vs Bottle+Box vs Bundle
• Tags: White Bubble vs Green Tag vs None

‍

CTR Results

The most impactful creative component from this experiment was the Video format., with an avg. CTR of 6.19%. The next highest performing creative component was the Product by itself, beating out both the Bottle+Box and Bundle offering.

‍

Click Results

In terms of clicks, we see again that Video is the top performing element collecting 16.8% of our total clicks. Image was not far behind with 16.5% of total clicks, however the Image CTR was much worse than Video, meaning Image is less expensive to run (in CPM). The worst performing creative components in clicks (just like in CTR) we're the Bundle product and the Green Tag.

After running five cycles and spending a little over $1,000, we have a total average CTR of 4.99%, and total clicks of 10,868. We've seen pretty steady performance across these experiments, so next month we'll need to try testing a wider variety of elements to try and beat our performance!