Does Creative Matter? The Meta-Experiment

About a year ago we had the idea for an experiment. A meta-experiment. A meta-meta-experiment. We wanted to learn two things:

1. How much tiny details in ads make a difference on performance
2. How fellow marketers respond to getting ads tested on them

Here are the results:

For image ads: The best ad outperformed the worst ad by: 10.33x Clicks, +548.71% CTR

Largest performance improvements from a single variable change:

• Text Layout: Using Center Middle text instead of Bottom Middle text leads to 5.16x more Clicks and a +315% increase in CTR
• Font Type: Sans-Serif compared to Serif font leads to 5.81x more Clicks and a +394% increase in CTR
• Font Color: Displaying White-on-Black as opposed to Black-on-White leads to 2.74x more Clicks and a 201% increase in CTR
  

For video ads: The best ad outperformed the worst ad by: 4.09x Clicks, +314.95% CTR

Largest performance improvements from a single change:

• Intro Scene: Showing a Solo Human compared to a Kitten leads to 3.46x more Clicks and a +201% increase in CTR
• Format: Keeping the headline Static as opposed to a Constant Headline leads to 2.75x more Clicks and a +215.83% increase in CTR
  

Fellow Marketers Loved the Experiments:

• For just $488 across 92 ads, we earned over 31000 impressions and over 2800 clicks

Comments fell into 3 (positive) categories:

1. Cool

• "Very very cool."
• "Cool little experiment!"
• "Looked into this. Really cool."
• "This is a really cool experiment 🔥🔥🔥"
• "This is a really really freaking cool idea"
• "This is really cool. Thanks for sharing the results! Is there a way to get in on any of your other studies?"

2. Meming

• "I hypothetically just purchased this product and may or may not be enjoying it right now"
• "hypothetically speaking, possibly. this isn't even a comment"
• “Hypothetically speaking, this is a meme."

3. Conjecture on why it worked

• 'It was the weird bottle’
• 'The product’s not real, but if it was I wouldn’t have stopped'
• 'The word experiment made me stop'
• 'Ooh shiny thing and/or Helvetica Font'
• 'FB glitched...'
  

To see the actual ads and performance data for yourself, check it out here: http://www.doescreativematter.com/archive/1‍

To comment and conjecture on the data yourself, tweet it out!

If you tweet, you’ll receive a special packet of data relevant to your industry 😉