Omnicom Digital's Jonathan Nelson on Evaluating AI

The adoption of artificial intelligence (sorry Trump, that's what it's still being called in these parts) has become the defining challenge and opportunity for agencies worldwide.

Few executives have been more closely involved in that transformation than Omnicom Digital's CEO, Jonathan Nelson, who has led the development and integration of AI across the holding company's Omni platform.

In the first installment of a new interview series featuring agency leaders and marketers, Nelson discusses the realities of driving AI adoption at scale, the lessons learned along the way, and why marketers can no longer afford to view AI as anything other than a core creative tool.

Where does AI feature in your average workday? 

Our point of view is AI is a tool that's like a partner for you. It doesn't replace you. It helps you do your work faster, better, with more context, and is always on there in the background, whether it's on your phone or on your desktop. It's really your partner.

What has been the hardest leadership challenge in introducing AI into a creative business?

Well, I think everybody looks at the technical challenge. Oh, you know, who's ahead: Anthropic, OpenAIm or Google? They're really obsessed with that. And by the way, I'm old enough to remember computers really coming into the workplace. When computers first came in, everybody looked at these beige boxes on their desktop and said, "What do I do with that?" Then they realised, "Oh, it allows me to communicate, or tabulate, or write, or keep track of things, or spell-check," right? You ended up not seeing it as this 2001-like monolith on your desktop, and more as a tool to help you get your job done.

AI is really no different. Like I said, it's your partner. The truth is, you've got to teach people how to use it, of course. But human behaviour is that most people don't like a lot of change, and I'm no different. I often walk the same way to work, stop at the same places, and make coffee the same way every morning. People don't like change, so I think it's not a technical problem as much as it's a people problem. But it creeps in, and then the light bulb goes off differently for everybody. It's, "Oh, this allowed me to do my job better or faster," or, "Wow, that's a really interesting idea. Let me work on that with my technical partner here." So companies change one person at a time.

What are you hearing from clients about the potential of agents handling their media? 

It's very early on, and every client is different. Some clients are leaning in and saying, "Oh yeah, you should be doing this," while others are saying, "Don't do this." But the truth is, whether it's buying or using agents to do planning, planning is a different story. Take the ability to access datasets. We've got Axiom and Real ID behind things, so identity and datasets, and a lot of proprietary data. Most clients are there. They don't care about the means of production. They just care about the answer.

When it comes to buying agentically, it's early days. We were the first ones to do it. I was actually involved in the first agentic media buy ever, and yes, there were a lot of people watching that money. I don't expect that's going to change. But you have this idea that the machine has waypoints, right? It checks in, and both sides are watching it and making sure nothing goes awry. Over time, though, this is how programmatic started as well. You start to trust it. The good thing with agents is there are also evaluator agents, which are just beginning to creep in, where you have AI watching AI and double-checking it, and then you have a person overseeing it. So we're very careful. Media is money management on some level. We understand our role.

What's the biggest misconception you’ve learned from Omnicom’s adoption of AI?

Oh, interesting question. Well, first of all, the capability-meets-craft piece is moving very quickly, so it's hard to pin down what it is and what it's capable of, because every two weeks it changes. Across OpenAI, Anthropic, Google, all the open-weight models, you've got new players and renewed competition. You've got the Chinese players too. It's hard to keep a scorecard because it's constantly changing.

There is one real constant, though, and I'm back to the people. People come up with lots of reasons why they don't do things. So I think the misconception is, "What am I supposed to do with this?" or "It's insecure," or "It's not deterministic," or "It's not fit for purpose." Or, "I tried to create an image and it's got seven fingers on it." Yeah, it used to do that. It used to be insecure. It used to create images with seven fingers — we've all had that experience. But when was the last time you created an image with seven fingers on it? We've fixed all these things.

So people get this perception, and it just takes a little while to adapt, which is what people are doing. I can't stress this enough: this is a people thing. People have got to embrace it. Robots are not going to replace you. You're not going to have a robot sitting at your desk — that's just not going to happen. Will we automate certain functions? Of course we will, just like we have with regular computers and lots of other technologies. We just need better ideas, faster, and better outcomes. That's what our business is about. These are just super-powerful tools that are going to allow you to achieve that. That's how I see it.

Can you share a recent project where AI unlocked something your team couldn't have achieved as quickly or effectively otherwise?

There's an idea called evaluator agents, and evaluator agents are exactly what they sound like. They're agents that evaluate other agents, outcomes or content. They can do almost anything AI can do. So they could be looking at every single output and screening it for things like: is the tone correct? Are the colours correct? Are there any inappropriate elements in there? Is it doing what it's supposed to do? They're essentially running everything through a filter.

We have evaluators in place today. Are they super sophisticated yet? No, not to the level they will be. But they're already useful. In media, for example: did the money go where it was supposed to go, and does it add up? There's your media use case. And by the way, does my evaluator agent match Google's evaluator agent? We have a checksum, right? Our evaluator says, "Did you get the money?" Google says, "Yes." We say, "Did it run?" Google says, "Yes." We say, "Can you provide proof?" It says, "Yes. Okay, let me double-check that." That can and will be done with machines, and it has to be, because we're buying literally hundreds of billions of pieces of content weekly. There was that Hugging Face example a few weeks ago. These are clever systems, and if you don't write them correctly, because they're goal-seeking, they'll do whatever they can to achieve the goal. They're even smart enough to try to cover their tracks. That's pretty sophisticated stuff.

But that's a frontier-model example, and not where the general marketplace is today. If you really dive into evaluator agents, the way you write them is to make things very black and white: did it do this, yes or no? If it didn't, you escalate it to a human immediately. One of the issues with that Hugging Face example, as I understand it, was that it ran amok and tried to cover itself up. By the way, there was an evaluator that caught it, and eventually a human was notified. That's how they figured it out.

With us, we have humans in the loop at many stages, whether you're building creative or buying media. You have what we call waypoints. It stops and says, "Check the work." A human has to go in and literally sign off on it, and then it keeps going.

Are there any projects or campaigns that have caught your eye where AI was crucial in their development?

Yeah, there's a ton. I knew this question was coming, and it's one I smile at because there are literally thousands of examples, most of which are hard to write about because they're quite boring. This thing's ability to do research and bring macro and micro context, datasets like Axiom or Real ID, into an equation on a superhuman scale is incredible. A lot of it is research, which I realise is not that exciting, or looking at marketplaces, understanding competitors and doing competitive analysis. Again, not that exciting, but critical to the craft.

When you look at the craft itself, there's a lot more AI going on than most people would suspect. I asked one of our companies about it and I spoke to BBDO and they said, funnily enough, they're using it all over the place. They have a funny little slide deck that talks about dogs, cats, and birds. For some reason they're doing a lot of pet-food work, and we've had a couple of cat-related campaigns. I don't know why, but cats seem to lend themselves to AI. One was Temptations, which I believe was the first AI-generated YouTube ad by a major brand. The second was for Whiskas, where BBDO did a cat casting and then used AI to create the backgrounds, clone the cats and produce a cute commercial.

Another example took a funny spin on AI. We all know that each AI has its own personality. They were playing with the fact that AI can sometimes give the wrong answer, which tied into the Snickers line, "You're not you when you're hungry."

The idea was: feed the AI a Snickers and it'll give you the right answer. Literally, you'd drop a Snickers bar into the prompt and suddenly it would come up with the right response. It's a little joke about AI, but AI itself is at the heart of the campaign.

So there are lots of examples. Some of the most important are behind the scenes, helping people research, analyse and create work faster. Others show up directly in the creative, like the Temptations, Whiskas, and Snickers campaigns.

If every creative professional could master one AI skill this year, what would you advise?

Well, first of all, I think it starts with curiosity, which is I know not a skill. It's just a trait. Humans by nature should be curious. Yes, prompting is a skill, duh. But look, just try it, right? And then look at what it does, and then you know, curiosity would lead you to say, "Well, did it do what it was supposed to do? Is this a good answer? Maybe I should retry it and see if my input changes the output." That's just curiosity, and I think that once you go on that journey, you realise that hey, what whatever I put in changes what comes out, and then you're on. Now you're on the path, and the more you do it, the better you get, and the closer you get to the ideas you want. You know, this thing really can be your partner, but your partner is not going to communicate back unless you ask it a question, whether that's a human or a machine.

We're trying to tap into that innate human trait, which is curiosity, and say, "Look, we can, yeah, we can give you tips, you know, but like, just use it, please, and then let's have a conversation about tips. Notice how I keep coming back to people. This is just a tool. It really is. It's no smarter than you are. It just can it can digest a lot more information, so it may look smarter, but it's not truly smart. It just you know the great thing about computers is they can ingest you know millions of rows of information and organise it and spit back an answer, right? Stuff that is inhuman, but that doesn't make it smart. Like a great creative with these tools, it's going to be a big deal.

There's actually a level of collaboration I've never seen before with some of our largest clients. The level of partnership is deeper than I've ever seen. We get shoulder to shoulder and they'll say, "What about this?" And we'll say, "Here's what we think, but we're not sure. Would you like to test that hypothesis?" It's not going to be cheap, but we need an answer because, if we can figure it out, there's real value there. I'm thinking particularly about things like GEO and model optimisation algorithms. How do you surface data inside models, and how do you change the outputs? That's changing all the time, and clients are really interested in it. The challenge is that the models themselves are changing, so the outputs change too. As a result, the level of collaboration and experimentation is quite significant. At the end of the day, it's groups of people working together shoulder to shoulder, agreeing on a common set of goals and then figuring out how to get there.

Our job is to bring tools, platforms, datasets, and knowledge to the table. This may sound a bit metaphysical, but we're on a journey together because it's all changing so fast. It's okay that we don't know the answer to every question.

Then there are other clients who'll put in a prompt, generate a video of a product and say, "Look, I can do your job." Well, if you hired a production company, you could have got some film too. When you really get underneath the output, the car doesn't quite look like the car. The details aren't exact. A lot of clients think that because they can write a prompt and 30 seconds later a video comes out, they've made a commercial. It's a little trickier than that. Which is fine. I can't stop people doing that. But I believe, and I think I'm being proven right every day, that AI is going to create an incredible amount of AI slop. It's going to create a sea of sameness because everything is trained on broadly the same pool of data, effectively the internet. If you ask similar questions, you're going to get similar answers, and everything starts to look the same.

What it can't do is replace true creativity. I actually think great creative minds are going to become even more valuable than they've ever been. It might take a little while because we're all enamoured with AI right now, but when everything looks the same, something genuinely original will stand out.

You're about to see this in music too. A lot of AI-generated music is coming out, but great musicians are few and far between. A computer isn't going to be Yo-Yo Ma, or Dolly Parton, or whoever your favourite artist happens to be. It can do something that's sort of like them, just as a generated video might sort of look like the car, but it's not the same thing. It isn't a great drummer or a great artist. People will figure that out, and I think they already are.