AI Read 100 Radio Ads and Found Short, Local and Emotional Wins

Happy Saturday. Welcome to the Marketing MadMen. This is going to be a unique episode. I don't know how many of you out there have seen Memento, but we're going to Memento this puppy. So we are with Jason Tatum of CallRail, not of the Boston Celtics. You'll catch up with that later in the episode. And what happened is we were initially recording segments two and three, but in that recording we came up with probably an ingenious, life-altering, changing of the continents idea for an experiment. We carried out the experiment, and we're going to start the episode by talking about the experiment. Pretty unique stuff, but stick with us and you'll be fine. So Jason, how you doing, bud?

I'm doing great. Happy to be here.

So we were talking, and this episode pointed us towards, in addition to everything else CallRail does, dynamic number placement and attribution and closing the loop, all the different things, AI came up. And it happens that their AI is audio based. And as a radio station, everything we do is audio based. So the quick version, and I will let Jason go into more detail because I'm going to botch it, is we ran an experiment on audio for commercial advertisers who have succeeded, and succeeded is broad, but mostly they've renewed, they stayed with these radio stations, and then ones that didn't. Now, there are a million variables as to why success did or didn't happen, but we tried to get the scale large enough, 40, 50 commercials, to get rid of some margin for error. I have not seen the results. So Jason is here to walk through the experiment, explain a little bit about how this technology worked, and then read some results from said experiment. So Jason, give it to us, brother.

Yeah, absolutely. Okay, so basically this is set up in two parts. You gave me a pile of a bunch of your episodes. The first thing that we did is we ran it right through our summarization method. Right, so we said, take all these episodes.

44 minutes.

44 minutes. Okay, so take 44 minutes of audio content, run it through the model and tell me in the most concise terms, four to five sentences max, what happened there, right? And so we have this one, let's see who this is. Jeff Hillmeyer started his first business building websites in college, got to 75 employees and sold it. He later started Dragon Army, an app studio, for 10 years before founding Purpose Group to buy and hold companies, la da da da da. Right? And so you get this sort of, I can spend 10 minutes knowing what happened on every single one of these episodes, and this all happened, you know, within about…

You're Cliff Noting versions, you're consolidating it down, but it's IP that in theory we own, because it was something we spoke of, we said. So it's taking it and making it concise, readable, and then highlighting what's consistent throughout, maybe what's missing potentially. It's trying to find trends and patterns.

Yeah, and what you really should understand is that if you were in this technology six, seven years ago, you would try to get a transcript, and it used to be you had to pay somebody to actually transcribe your voicemails and things like that. It was a human that did it. And then there started to be AI technology that was able to try its best, and it was laughably bad, right? We would sit around and read transcripts and be like, oh gosh, this sounds bad.

And what you had to do was get a loose transcript and have someone fix it, which is counterintuitive anyway, because then you're just paying somebody anyway, and sometimes it's even harder to do it that way.

And then, without getting into deep tech speak, the world changed several years ago with a whole new way of doing things in the large language model world. And now we get things like these summarizations that it just sounds like a human wrote it. Like, that's how good it is, and it understands what you need to know. It understands how to be concise. It's truly brilliant.

And it is being objective, not subjective, correct? It is saying fact that is pulled out. It's not saying this was good or bad. It's just looking for patterns and catching things, correct?

It can do that. That's what the summarizations do. But while it is objective, as we get into some of your more effective and ineffective ads, you're going to find that it does have some opinions.

Let's hear the opinions. Oh my God. It's funny, we've been having a lot of conversation around the office about objective or subjective. And we've run a really damn good company being subjective. But you know, there are some times we've got to strap those boots up and it's objective time.

That's right.

Numbers are objective. Ones and zeros are objective. So I'm curious to see how this goes. So give it to me.

Well, let me tell you what the ones and zeros said about your effective ads. We'll start positive. Effective ads. You gave me, I think, 50 of each, right?

Something like that.

So we analyzed 50 in our model and it came back with: the effective ads are concise and to the point. They establish credibility. They're local, trustworthy businesses. They make an emotional connection. They speak directly to the interests of the target audience. They use humor and inside references tailored to the audience.

Wow.

They promote limited time offers and deals, which people like, and focus on benefits and solutions for the listener.

So for the record, and I'm not tooting my own horn here, marketing 101. All of those things are elements that require you to do something and elicit some sort of emotional response. And ironically, data can't elicit emotional response. It's why radio is a good medium. You can do imagery. You can do inside jokes, right? It's hard to do that with internet advertising. So that's amazing. What about the not so good?

Not so good. Okay, so I hate to tell you this. They're too long and wordy, and they contain irrelevant details.

Love it. As a marketing guy, I love it, because some of this stuff you innately know, right? That's why they didn't succeed. If you're wordy or keep going.

Pushy sales pitch, overly promotional language.

Interesting.

How does that make you feel?

It makes me feel like we know why they canceled, and that we stick with the ones that work. So look, the goal of this was exactly that. So when we meet with clients, we're like, look, we did this experiment. We know that this is what works in commercials. So how do we write commercials that elicit these responses or do these things?

Yeah, exactly.

And that is the plan. So let's give you guys some, how else would you apply this? In your business, when you're meeting with people, when you're talking about marketing attribution and making sure marketing is working, how do you apply these techniques? Because I want to make sure you guys get credit due here, because this is not simple stuff, but this is something that can be incredibly powerful for most businesses. Talk about how you'd use data sets. It doesn't have to be these exact ones, to accomplish a goal for a marketer.

All right, so you're a marketer who is held accountable to the leads that you drive, right? You're driving leads, you're driving quality leads that turn into dollars and cents, real good customers, right? Whatever that looks like, whatever your business, whether you're a plumber, whether you're a marketing agency working for Main Street, you're a lawyer, a doctor, it doesn't matter, you know, the candlestick maker, it's all the same thing, right? And so typically a good marketer is trying to optimize their ad spend to get leads the most efficiently, right? So you're not just trying to get leads, you're trying to get good leads. You're not just trying to get good leads…

Bad leads waste the time of the people on the call. They cost money from certain sources. And if they're not coming in, no one does anything. So you're trying to optimize the lead sources to be good leads?

Yes. And here's the thing: in all businesses, to a certain extent, unsustainably, but to a certain extent, you can buy leads. Right. You can buy leads, but…

In the digital world, you can buy anything. Views, listens, reviews. I mean, anyone that thinks otherwise is crazy. But most people have gotten discerning enough to tell the difference.

Yes. And what you want is, ultimately, leads, but good leads, but efficiently, right? And the efficiently is super important. So what happens is a good marketer, especially those that are trying to drive conversations and understand from the conversations what worked or what didn't work, they're also listening for why did they convert? Why did they not convert? And then they realize, oh, the guy on the phone is botching it, right? They're not doing what they're supposed to do. They're not saying what they're supposed to say.

So it wasn't necessarily that the lead was bad. It was that what was used to convert the lead was bad. Something in the chain is broken. You're trying to find what part of the chain is broken.

Yeah, there's a funnel, right? And where is it breaking? And so whether you're a business owner, whether you're the marketer in a business or an agency, you usually are going to want the same thing. You're going to want visibility. You're going to want insight. And what it used to be was, okay, great, you get on a phone call, I'm going to pick up and listen in, right? And then VoIP technology came around and you started to be able to monitor calls. You could even whisper in coaching, right? But what people found was, I don't have the time to monitor enough calls, right?

And most importantly, this is important for scaling your business, right? If you have three people answering the phone, four texts, you could probably do that alone. But to scale your business, these are systems you need to implement to be able to scale the business in the first place.

That's right. And so then you say, okay, can I record the call? Right? Okay, when I record the call, I'll listen to them in my own time. I'll do it at 2x speed, yada, yada. But even that human is only able to listen to so many phone calls efficiently, right? There's no scalability there without filling up a whole farm of desks.

And I also imagine if you're not doing this in real time, something is lost. Because if you're doing it three weeks down the road, so much could have changed in those three weeks that your margin for error just increases on what really is effective and what is not.

That's exactly right. And so eventually that leads to, well, can I get a transcript? Okay, can I get a good transcript? Okay, and now you still are limited by how many transcripts you can read. How many transcripts can anybody read, especially anybody with any knowledge of the business?

Now you have the data laid out, but you have nothing processing it. It's the time that's required to process it. No offense, laying out data is easy. Processing is not.

Yes. So what if there was technology available now that does all of that for you in an instant, at scale, right? What if you could analyze 100 of your ad campaigns and get back the insights that you need to understand what worked, what didn't work, and what maybe you should do, right? That's the world that we're entering, right? And some of this can be like coaching an agent, right? And it's one thing when your manager sits down and tries to coach you, and you may think that your manager hates you and that there's bias wrapped around this, but it's different when this is ones and zeros.

Right, this becomes objective. You make this the fact, because this is proven by a sample size. And again, the machines can't be subjective. They can't dislike you because they don't like you.

Yes. So I want to caveat one thing, because you brought it up, marketing 101. If you're a good marketer, this stuff is intuitive. It's the kind of stuff that if you were to go to a big ad agency's website and look around, you might have to put your email in to get a white paper to read some of this stuff, right? If you've been in the game long enough, it's just second nature. But people go to business school to learn this stuff, right? So imagine, this stuff is second nature to somebody who's tenured and knows what they're doing. But then we take, at scale, 100 episodes, 100 ads, feed it through, and the model says: based on these examples, effective ads for this audience should be short, simple and conversational. Establish trust and credibility as a local business. Appeal emotionally with humor, sports references, etc. Promote special deals, focus on benefits and solutions for the listeners' needs, and avoid overly salesy, pushy language. And above all, target sports fans specifically with relevant messaging. I love it. The most effective ads seem to come from local, credible businesses and connect with the audience through an emotional appeal, shared identity, humor, etc.

I love it. And one of the things, and it's so nuanced, but notice they didn't say discounts. Special deals. Okay, here's the biggest problem you have in a lot of marketing. They go to you and they say $100 off. It's like, oh my God, but if you go to the website, it's $100 off. You don't feel special at all. So a special deal is only for 680 The Fan listeners, blah, blah, blah. So there is a way to delineate that. And that's important because there's a difference between a coupon waving person and someone who's looking for a special deal. Those are different things. They're very different things, right? So I cannot believe that's what it spit out. And if I was to just throw out stats that I think, I've been doing this for 25 years, I would probably throw out a lot of those things. And one more caveat I think is important. Your outputs are as important as your inputs. And I took a lot of time to discern what I believed was good performing and bad performing. So if you went into this just throwing stuff against the wall, it is going to be less functional. But once those parameters are set, you're easy for the future, right? If every commercial you do, you just drop into one of two buckets in real time, that takes two seconds. Then all of a sudden you have these huge data sets of inputs that you can turn into outputs.

And I want to make a converse point. You spent time and energy to curate this the right way. You could have gotten all these insights yourself, with 25 years of experience. You would have had to go back and listen, take notes, do all the things. That would have taken you a couple hours, probably.

Way more than that.

So we did it in minutes. And the important thing to know is that if you're listening to this and you're like, this sounds like he just bullet pointed this stuff out, it sounds like a human wrote it. It doesn't sound like Clippy in my Microsoft program. We did not edit this. It is not doctored. And the other thing that's really important is that we didn't even really try that hard to get any particular response out of it. We did kind of the bare minimum with it and got this output. And so what you're going to see, very quickly over time, is that even the most generic prompts are going to get smarter and smarter and smarter. And then when you know what you're looking for and you can ask more intelligent human questions of the models, you're going to get more and more insightful information that's going to help you run your business better. And it's going to save you time and money and give you a lot of the things that now take a lot of time and money to do.

Honestly, Jason, this has been awesome. We're going to send out a lot of content about this. I'm a show your work guy, so I'm going to show the spots we did. I mean, this was a real experiment. And honestly, we will show the method that went behind this, because again, you're right, it's easy to just pretend like you did this stuff. So check LinkedIn, check Facebook. We're going to put a lot of this data out. You're probably hearing about it first. We're going to release this stuff after the fact. But you've been listening to the Marketing MadMen on Xtra 106.3, and we'll be right back.