Is it time to rethink pre-testing?

Visitors to LinkedIn over the past few weeks will likely have seen an interesting debate - and we're not talking about the posturing over the Jaguar relaunch ad.

Instead, various discussions focussed on the efficacy and relevancy of pre-testing, and was sparked by System1 declaring its Christmas retailer "winners" long before the tills finally closed and the store shutters camne down on Christmas Eve. This post was particularly juicy, and eventually prompted some of the trade websites to deliver their hapenny'worth.

Pre-testing is nothing new of course - marketers have often looked for a tool to justify expenditure on new campaigns to their boards. For agencies, however, they tend to be less popular - some testing methods apparently rely on registering positive emotions in ads by pressing smiley face emojis during viewing.

Other emotions are, of course, available other than smiley faces, as the LinkedIn discussion showed. Given that the pre-testing debate has produced a lot of heat but not a lot of light, we're asking whether it's time to give the technique a re-think.

Jo Arden, chief strategy officer, Ogilvy UK

The annual pre-test tear-up is here and this year it’s bigger than ever, here’s why:

1. Outrage at crowning brands ‘the most effective’ before a single sprout has been sold. Fair enough, though hopefully the audience (as in us) know that pre-testing is only an indicator, not a measure, of impact.

Pre-testing is done with people whose money we’re after, so it’s a better measure than your industry mates giving it the Emperors’ thumb.

Maybe the real issue is:

2. We’re oddly sniffy about self-promotion and think our hard-crafted ads are cheapened by the pre-test rankings. The old double-standard is firmly at play if we begrudge our research partners leveraging a bit of festive marketing.

To that I say – good luck in your agency-of-the-year-submissions/awards entries/LinkedIn posts. Merry Christmas, may we all be humbled and proud!

Ed Hayne, strategy director, Grey

Pre-testing Christmas adverts and plastering the results all over LinkedIn is entertaining but undermines the great strides that have been made in recent years. All the limitations become amplified, and the snake oil salesmen are out in force.

Of course, ‘gaming’ the system isn’t anything new, but the festive season seems particularly ripe for exploitation. For example, agencies are well aware that simply advertising ‘Christmas’ (something most people already love) should enable them to sail through any pre-testing that focuses on emotional response.

However, away from these shortcomings, the fact that there’s a growing bank of evidence which proves its worth, undermines the dinosaurs who want their gut instinct to hold greater sway when huge budgets are being signed off.

Whilst no tool is a silver bullet, and it’s dangerous to think of them as that, now is the time to build upon the excellent work that’s been done of late, not re-think it.

Mike Treharne, head of data & insight, Leo Burnett

50 years after the publication of Alan Hedges' seminal critique Testing to Destruction, this question remains as relevant as ever.

While it’s entirely reasonable for brand owners to seek evidence of a campaign’s commercial impact before signing off substantial production and media budgets, pre-test providers can be guilty of overstating their tools' predictive power. There is precious little in the marketing science archives that demonstrates a causal link or even correlation between pre-test metrics and ROI or financial outcomes.

The term "test" itself suggests a definitive performance measure which does these tools a disservice, since they do have genuine diagnostic value. Acknowledging this and emphasising their complementary role alongside other inputs, including subjective instinct, might enhance their credibility. Perhaps it’s time for a rebrand - pre-yardstick, anyone?

Racquel Chicourel, chief strategy officer, TBWA\London

To pre-test or not to pre-test? The truth is, like it or not, pre-testing serves a noble purpose: helping with alignment, mitigating risk, and unlocking sign-off from senior stakeholders.

Pre-testing platforms today are the best they've ever been. The issue is not the quality of their method or output, the issue is when we test everything and treat the results as gospel. That's when these platforms shift from something great to something dangerous to creativity. Why? Because some of the best creative ever— Cadbury 'Gorilla', Coca-Cola’s ‘Hilltop’, Budweiser 'Whatsssup' and Guinness 'Surfer'—failed pre-testing.

These campaigns were not just commercially successful but made these brands iconic and set new creative standards for our industry. So, let’s embrace pre-testing but always leaving space for the wild cards that defy convention. As a marketing community, let’s all be open to taking leaps of faith with ideas that might never get the pre-testing pass.

Jamie Elliott, chief executive officer, The Gate

Most ads that make it to screens—yes, most—are rubbish. You do not have to watch many pre-rolls or ad breaks to realise that there is a sea of ads out there with endless voice-overs and poor production values, without much wit or evidence of a creative idea. These ads weren’t ruined by pre-testing; in fact, pre-testing could only help improve them. Widespread pre-testing during development might elevate the industry’s baseline, resulting in fewer sub-par ads overall.

Yet, the pre-testing hype has spiralled into something almost evangelical. These tools—just tools—are marketed like religions, complete with believers, gospels, and clever Christmas ad post-testing to spread their doctrine. It’s a brilliant marketing strategy but risks obscuring the truth: pre-testing tools are just predictive mechanisms with inherent limitations.

They excel at measuring specific emotions but struggle with understanding how to maximise cultural impact. They’re effective for traditional TV formats but less so for non-traditional, innovative shapes. Like all tools, they will never be good at everything you want them to be good at, they’re good for this, but not for that. We lose sight of this at our peril.