If you’re running paid media with an external agency and you’re not sure whether you’re getting a fair deal, you’re not alone. Most clients reach the point of questioning their agency after months of ambiguous reporting, slow creative cycles, or declining results that are always explained away. At Choco Media, we’ve been on both sides of this conversation — and the questions that reveal the most about an agency’s quality are rarely the ones clients think to ask. This post is a structured framework for your next paid media agency audit: 12 questions, organized by category, that will tell you whether your agency is doing the job.
This isn’t a post about how to fire your agency. It’s about how to evaluate one honestly — including the possibility that they’re doing well in areas you weren’t measuring. A good review surfaces both problems and strengths. The goal is clarity, not confrontation.
Whether you’re preparing for a quarterly business review, considering a switch, or just trying to get more out of a relationship that feels stuck, these questions are the place to start.
Why Most Paid Media Agency Reviews Fall Short
The typical client review of a paid media agency focuses on ROAS, CPL, or CPC — and stops there. Those metrics matter, but they’re not sufficient. An agency can hit your ROAS target for three months by reducing reach, pulling out of competitive placements, or retargeting a warm audience into exhaustion. The number looks good. The business isn’t growing.
Conversely, an agency might be doing genuinely sophisticated work — building prospecting funnels, testing creative systematically, building up exclusion lists — and temporarily showing weaker numbers during a growth phase. If you evaluate only on short-term performance metrics, you may misread both scenarios.
- Vanity metrics (ROAS on retargeting) can mask top-funnel weakness
- Attribution windows set too short inflate last-click results
- Agencies sometimes optimize toward what they can measure, not what matters
- QBR presentations are often prepared to reassure, not inform
A structured review asks harder questions. The 12 below are organized into four categories: performance, process, transparency, and fit.
Performance Questions
1. Are we measuring the right things?
Before reviewing performance, confirm that the KPIs being tracked are actually connected to business outcomes. An agency that optimizes for platform-reported conversions without cross-referencing CRM data or revenue can show great numbers while contributing little real growth.
Ask your agency: how do we reconcile platform data with what actually shows up in our CRM or revenue report? If they don’t have a clean answer — or if the two data sources have never been compared — that’s a gap worth closing before any performance review.
- Are we tracking view-through conversions? What’s the policy on those?
- Is our attribution window consistent across channels?
- What’s the lag between ad click and conversion for our typical customer?
2. What does new customer acquisition look like, separately from retargeting?
Many agencies bundle prospecting and retargeting results together. The ROAS on retargeting is almost always high — you’re showing ads to people who already know you. The harder work is bringing in new customers. Ask your agency to break out new-vs-returning customer performance explicitly.
If this split isn’t already in your reporting, that’s a structural gap. It should be. In client work we’ve found that agencies which actively segment and report this way tend to have a much clearer view of what’s actually working at the top of the funnel.
3. How much creative is being tested, and what does the test structure look like?
Creative quality is the primary driver of paid media performance in 2025 and beyond. Targeting has narrowed as platforms automate audience selection. What’s left for humans to control is the message, the hook, the visual — and the testing process that identifies what works.
Ask to see a structured breakdown of live and recent creative tests. Not a list of ads — a test structure, with a hypothesis and a result for each. An agency running genuine creative testing will have this. One that’s running ads without a framework won’t.
- How many creative variants were tested in the last 90 days?
- What were the top 3 findings from creative tests this quarter?
- What’s the testing cadence — how often are new concepts introduced?
For a deeper look at how we approach this, see our post on how to build a paid media testing budget.
The agencies that consistently outperform over 12 months aren’t the ones with the best initial setup. They’re the ones with the most disciplined learning process — testing more, learning faster, compounding those learnings into creative direction.
Process Questions
4. What’s the brief-to-launch timeline for new creative?
Speed of creative iteration is a competitive advantage. If your agency takes four weeks to brief, produce, and launch a new creative concept, you’re running about 13 tests per year. If they can do it in 10 days, that’s 36. Over a year, the faster team learns three times as much.
Ask for the actual average timeline from brief to live ad in the last quarter. This is a concrete, measurable number. If the agency doesn’t track it, that’s telling. If the number is over three weeks for digital creative, ask what’s slowing it down and whether it can be reduced.
5. Who is actually working on our account, day to day?
It’s common for agencies to pitch senior strategists and then hand the account to a junior media buyer. Neither seniority level is inherently wrong — junior buyers can be excellent, and senior strategists don’t need to be in the platform every day. But you should know who is making decisions and what their experience level is.
- Who handles day-to-day optimizations?
- Who writes the creative briefs?
- Who is accountable when something goes wrong?
- How often does a senior strategist review account performance?
6. What’s the process when something underperforms?
Every paid media account has underperforming periods. The question isn’t whether your agency encounters problems — it’s whether they have a structured response to them. Ask them to walk you through a specific example from the last six months where results dropped and what they did.
A good agency will describe a concrete diagnostic process: what they checked, what hypothesis they formed, what action they took, and what happened after. An agency without a clear process will give a vague answer about “optimization” or “platform changes.”
Transparency Questions
7. Do we have direct access to our ad accounts?
You should always have admin access to your own ad accounts — Google Ads, Meta, LinkedIn, whatever platforms you’re on. This is non-negotiable. It’s your data, your audience, your historical performance record. If your agency is managing accounts they own rather than accounts you own, that’s a structural risk. Switching agencies becomes significantly harder, and your data history may not transfer.
If you don’t currently have admin access, ask for it. If there’s resistance, that’s a red flag.
8. What’s included in the management fee, and what’s billed separately?
Agency pricing varies widely, and not all fees are disclosed upfront. Common additions include creative production costs, platform fees passed through with a markup, third-party tool costs, and additional charges for additional campaigns or channels.
- Is there a markup on media spend? What percentage?
- Are creative costs included or billed separately?
- What triggers additional fees outside the retainer?
If you’re evaluating or renegotiating a retainer, our campaigns page outlines how we structure our own pricing — which may be a useful reference point.
9. How is our budget actually being spent?
Ask for a budget allocation breakdown: by channel, by campaign type (prospecting vs. retargeting vs. remarketing), by audience, and by creative. Most agencies can provide this. The ones who can’t — or who present it in a way that obscures the detail — are hiding something or haven’t built the infrastructure to track it clearly.
Also worth asking: what percentage of our budget goes toward platform fees versus actual media spend? If the agency is using significant third-party tools charged against your budget, you should know that explicitly.
Strategic Fit Questions
10. What do you know about our industry that you didn’t know six months ago?
A good agency learns your industry over time. They should be getting smarter about your competitive landscape, your customer psychology, your seasonal patterns, your positioning relative to alternatives. If their strategy looks the same after 12 months as it did at the start, something is missing.
This question is harder to fake than a performance number. Ask it in a meeting, not in writing. Watch for specificity — good answers will reference something concrete about your business or sector. Vague answers about “digital trends” suggest the team isn’t paying close attention.
11. What would you do differently if budget were not a constraint?
This question surfaces strategic ambition and honest thinking. If your agency’s answer is just “spend more on the same things,” they’re not thinking strategically — they’re executing a template. If they describe a new channel, a different creative direction, a testing program they’ve been constrained from running, or a different funnel structure, that’s a more useful answer.
Even if budget doesn’t change, this question tells you whether your agency is thinking beyond the current plan.
12. What are we not doing that you’d recommend to a similar client in our position?
This is the most direct version of “are you holding back.” A confident, strategically engaged agency will have recommendations they haven’t yet pushed hard enough — a channel worth testing, a landing page hypothesis, a creative format, a targeting approach. An agency that can’t answer this question may have run out of ideas, or may have stopped thinking proactively about your account.
- Are there channels we’re not on that you’d explore given our audience?
- Are there conversion rate improvements upstream from the ad that we’re missing?
- Is there creative territory we haven’t tested that you’d want to try?
For a complementary view on how landing page quality affects paid media results, our conversion rate optimization service page covers the intersection in more detail.
How to Run the Review
The most useful format for a structured agency review is a live meeting, not a written questionnaire. These questions are designed to be asked in conversation — partly because the quality of the answer depends on how it’s delivered, and partly because follow-up questions often yield more than the initial response.
We suggest running this as a 90-minute QBR with a set agenda shared in advance. Give the agency the opportunity to prepare — but ask for specific data, not polished decks. The goal is a real conversation, not a presentation.
- Share the question categories (not the exact questions) a week in advance
- Ask for raw data exports alongside the deck — platform reports, creative test results, budget breakdowns
- Have at least one person in the room who can read the data independently
- Take notes on both what was said and what wasn’t answered clearly
What to Do With the Answers
After the review, you should have a clearer picture across four dimensions: performance quality, process maturity, transparency, and strategic fit. An agency can be strong in some areas and weak in others — and your decision about what to do next should reflect that nuance.
If the review reveals specific gaps, the first step is usually a direct conversation: here’s what we found, here’s what we need to change, here’s the timeline. Many agency relationships that feel stuck can be improved significantly with clearer expectations and a reset conversation.
If the gaps are structural — no account access, opaque billing, no real strategic thinking — those are harder to fix without a change in the relationship or the agency itself.
If you’re at the point of evaluating alternatives, we’re happy to talk through what a well-structured paid media engagement looks like from our side. Reach out via our contact page and we can be specific about approach, scope, and process.