Beta

How VWO Uses GPT-3

To generate copy variants that beat their human counterparts
Brett Friedman

The Company

VWO is like Marpipe for websites. With VWO, you can run a/b/n and multivariate tests on any part of your website, including in-app (if you have one). They also happen to be the industry leader with over 2500 customers across 90+ countries.


The Experiment

OpenAI's GPT-3 API is an AI that can perform "any language task - semantic search, summarization, sentiment analysis, content generation, translation, and more." That open-ended "and more" is the exciting part. My first thoughts went to copywriting.

And while others are making the news for their creative uses -Gwern's poetry, a college kid's #1 HN post, a business idea generator, and a controversial Guardian article written completely with GPT-3 - VWO immediately moved to test the AI against us lowly humans in a game of making money with words in verticals as far ranging as SaaS to Travel, Education to Logistics, Finance to Automotive.

You can see the official rules below, but the key (besides VWO's brilliant marketing play) is to test the % improvement of website copy with a clear measurable goal.


The Results

With 13 confirmed participants, 4 results are currently in and the early numbers fare well for our robotic counterpart, Mr. GPT-3. In two contests, GPT-3 emerged as a clear winner. What's interesting, is that one test was a traditional A/B test while the other was a full-blown Multi-Armed Bandit showing early signs of the AI's sheer adaptability. You can see the comparisons below.

The other two contests, VWO ruled "inconclusive" because both companies stopped the test midway. Perhaps I misinterpreted the situation, but I'd argue that the humans behind those company's copy were embarrassed to be losing to an AI and paused the campaigns to avoid any negative PR backlash. But for now, we'll have to wait another few months to see how the competition pans out.  

The Application

If by any chance you can get your hands on GPT-3, DO IT. Just do it. If you're a copywriter, the earlier you learn how to wield AI like a wand, the sooner you'll overtake your competitors.

Boost ad performance in days with a 7 day free trial.
Claim Trial

How to Run a Multivariate Test

The Beginner's Guide

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Tiffany Johnson Headshot

How to Run a Multivariate Test
The Beginner's Guide

Plus, Get our Weekly
Experimentation newsletter!

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Outliers is the weekly newsletter that over 10,000 marketers rely on to get new data and tactics about creative testing.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Sign Up to Outliers 💌

Join 7,000+ marketers in the world’s best newsletter all about catalog ads!
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.