When Your Agency's Numbers Look Too Good to Be True
You brought on a new agency a month or two ago, they're fully up to speed, and suddenly every KPI is green and up and to the right. Before you relax, here's the part most business owners skip: the first instinct should be skepticism, not relief.
Let me describe a situation I get called about often. You brought on a new agency. It’s been a month, maybe two. They’re fully up to speed, they’re running with the account, and the numbers come back looking fantastic. Everything is green. Everything is up and to the right.
Here is the uncomfortable thing I have to tell people: that is not, on its own, good news. A genuine improvement at the account level usually shows up unevenly – some campaigns move, some don’t, some metrics improve while others stay flat because they were already fine. A clean, uniform sweep of green across every metric in month one or two is, more often than I’d like, the signature of someone optimising the report rather than the account.
So before you relax, break the picture into three layers and look at each one separately. They get progressively harder to fake, and progressively more important.
Layer one: the Google-facing metrics, everything before the click
The first layer is what I call the Google-facing, pre-website-visit metrics. Impressions, clicks, click-through rate, impression share, average position – everything that happens before, or at the exact moment of, the click. None of it tells you whether a single dollar turned into a customer. It tells you what happened inside Google’s auction, and that is a different thing.
The headline metric in this layer is click-through rate (the share of people who saw your ad and clicked it). A lot of business owners have been trained to believe that CTR going up is unambiguously good. It is not. At a high level, as a measure of whether the campaign is working, CTR is close to useless – and worse, it is one of the easiest numbers in the entire account to manipulate.
Here is one way I can double your click-through rate tomorrow without improving anything. Take a search campaign. I cut desktop bids hard, so the lion’s share of impressions and clicks shift to mobile – and mobile tends to carry a higher CTR. At the same time I bid up on mobile so the ads sit even higher on the page, which pushes CTR up further still. To cover my tracks, I keep the blended average cost-per-click roughly where it was last month. Now I can sit in the report meeting and say: “We didn’t raise bids, we didn’t change where you’re ranking, this is just our work taking effect.” The blended numbers back me up. It’s only when you segment by device, or look at the bid adjustments, that you see mobile was quietly cranked.
The same trick has a display-network version. I open the display ads up to the mobile-gaming app network – iOS and Android games. If you’ve ever played a mobile game, you know how often you tap an ad you never meant to tap. Those misclicks count as clicks. CTR goes up, cost-per-click often goes down because that inventory is cheap, and on paper it looks like a win. It isn’t. The traffic is accidental and the placements have nothing to do with your industry.
The tell for both is in your actual analytics – GA4, Adobe, whatever your real source of truth is, not the Google Ads interface. Pull the sessions from that campaign. The volume may be up, but segment one level further: average session duration is probably close to zero seconds. The pixel fired, the visitor landed, and they backed out immediately because they were never trying to be there.
Layer two: lead volume and cost per lead
The second layer is more transparent to you as the owner, because it’s closer to your business: lead volume and cost per lead. It’s also gameable on exactly the same principle – it all comes back to the quality of the traffic being driven.
A fact about Google Ads traffic that surprises people: there is a lot of spam, and a meaningful share of that spam will fill out a form. So if your form fills jumped from 10 a month to 40, and on the internal call everyone is scratching their heads because the marketing-qualified pipeline and the closed-won numbers didn’t move, you are looking at the same maneuver as layer one. Open up broad placements, or broad keywords with a high tendency to convert on paper, and the cost per lead in the Google Ads interface comes down while lead volume goes up. The key phrase there is cost per lead – not cost per marketing-qualified lead, not cost per closed-won deal. It’s the cheapest, fastest, most flattering version of the metric, and it’s the one that renews retainers.
This is the same problem covered in Red Flag #6 – nobody is tracking what happens after the lead form gets submitted. A form fill is not a customer. If the only number on the slide is cost per lead, you’re being shown half a metric – the convenient half.
Layer three: revenue and attribution
The third layer is the one people assume is un-fakeable. You can’t just invent a sale. That’s mostly true – but you don’t have to invent revenue to make paid look like it produced it. You just have to re-credit revenue that was already going to happen.
Here’s the macro tell. A new agency takes over, and reported paid revenue climbs, and keeps climbing. Pull back and look at the whole site. If paid is going up and organic is falling at roughly the same rate, a red flag should be going off. That is not incremental growth. That is the same demand being re-labelled. There was no new revenue created; credit was moved from one channel onto another.
There are a couple of ways that happens. One is changing the attribution model (the rules that decide which touchpoint gets credit for a conversion) so that organic and direct lose their share and paid absorbs it. The other lives inside the Google Ads attribution window (how long after a click Google will still credit a conversion to that ad). Most business owners assume a conversion means: user clicked the ad, then converted, right then. It doesn’t. The standard setup has a click-through window – I click an ad today, don’t convert, leave. Two weeks later I come back – by email, by typing in your URL, however – and fill out a form. Depending on how attribution is set, Google Ads can take full credit for that conversion.
Sometimes that credit is fair. Something can stick in someone’s head. But where it stops being fair is when an agency stretches that click-through window to its 90-day maximum. Ninety days is a long time. Can you name an ad you clicked three months ago and then acted on – for anything short of a major purchase? There are real arguments for longer windows: the click fed a remarketing sequence, an email nurture, a genuine multi-touch journey. But if your agency isn’t actually running that orchestrated journey – tracking the prospect through funnel stages and serving the right message at each one – then claiming the credit for it is just window-stretching.
I’ve written this one up as its own entry: Red Flag #12 – paid revenue climbs while organic quietly falls by the same amount.
The questions to ask
You don’t need to run the account to catch this. You need to ask the report to defend itself.
- Segment the Google-facing metrics by device, and show me the bid adjustments. If CTR jumped, this is where you see whether it came from the work or from a device shift.
- Reconcile the leads against the CRM. Of last month’s form fills, how many became marketing-qualified leads, and how many closed? If volume is up but qualified pipeline is flat, the new leads are noise.
- Pull the blended channel view – paid against organic and direct. If paid rose and organic fell by a similar amount over the same period, ask what actually grew.
- What is the attribution model and the conversion window set to, and did either change when you took over? A straight answer exists. If you don’t get one, you’ve found the layer that needs the most attention.
Good numbers can absolutely be real. But “real” survives all four of those questions. “Too good to be true” usually falls apart on the first one.