Proving AI Marketing ROI Without Attribution Data

Somebody above you wants proof that the AI spend worked, and your attribution data cannot provide it. The tools changed how work gets done, not which campaign a conversion gets credited to, so the channel report looks the same as last quarter and the finance question hangs there unanswered.

You are not behind the curve on this. In Jasper’s State of AI in Marketing survey of 1,400 marketers, the share who said they could prove AI ROI fell from 49% to 41% year over year - adoption spread faster than measurement did. My argument in this post: stop asking attribution a question it cannot answer, and build the proof from three things that need no attribution at all - baselines, holdouts, and honest confidence labels.

Why attribution can’t answer this question

Attribution allocates credit for conversions across recorded touchpoints, and AI’s contribution mostly is not a touchpoint. A model that cut your drafting time in half appears nowhere in a customer journey. Neither does the brief that made a post better, or the AI answer that recommended you without a click. The instrument measures paths. AI changed the factory. And that holds under every model - multi-touch or last click only changes how the recorded paths divide the credit.

The industry data says the instrument was already struggling before AI arrived. In TransUnion’s measurement research with EMARKETER, a July 2025 survey of 196 US marketing professionals, only 62% report even “some confidence” in their performance metrics, with siloed data (49%), cross-channel deduplication (48%) and walled-garden reporting (41%) as the top barriers. I read that survey as permission: nobody’s attribution is clean, so a proof built without it is not a workaround. The same survey found 67% now prioritize incremental ROI - the exact reframe this post is about.

Level one: the workflow baseline

The cheapest proof is a before-and-after on the workflow itself, and it is where I always start because it needs nothing but a note of what “before” looked like. Pick one workflow AI changed - the weekly newsletter, ad variant production, brief writing - and write down three numbers from before the change: what it cost in hours, what it produced, and one quality signal (open rate, review-pass rate, ranking). Then measure the same three after.

This is the evidence layer of my AI ROI calculation, and its rules carry over unchanged: saved hours only count if they were redeployed into something, and a lift only counts against a baseline measured the same way. What you get is not channel ROI. It is “this workflow now costs X less and produces Y more,” which is a sentence a CFO can check.

The limit of level one is that it cannot see cannibalisation or externalities. Faster drafting that produced worse posts shows up late, in the quality signal. That is what level two is for.

Level two: the holdout test

A holdout is the simplest honest experiment there is: keep AI out of one comparable slice of the work, run it in the rest, compare. No attribution model, no tracking pixels, no data science hire. The comparison is the proof, which is why incrementality testing survives every kind of tracking loss - and why I push it on teams who think experiments are an enterprise luxury. This section covers the shapes. The step-by-step design work - sizing, split choice, pre-registered readout - is in my incrementality testing walkthrough.

Three holdout shapes a small team can actually run:

The list split. Email is the easiest place to start because you control the split. AI-assisted subject lines and copy go to a random half of the list, your normal process to the other half, for a full month of sends. Same offers, same timing. The delta in opens, clicks and unsubscribes is AI’s measured contribution, full stop. Every email platform can do a random split today.

The content cohort. Split the next quarter’s planned posts into two comparable groups by expected difficulty, produce one group with the AI-assisted pipeline and one without, and compare cost per post and performance per post after ninety days. Imperfect - posts are never perfectly comparable - but a ten-post cohort beats an anecdote about one article.

The geo or segment holdout. Where AI touches a campaign (creative variants, audience copy), run the AI version everywhere except one comparable region or segment. This is the same geo holdout logic that survives zero-click measurement, pointed at a different question: not “does marketing work” but “did the AI version outperform ours”.

Three rules keep any of these honest. Change one thing - if the AI rollout lands the same month as a redesign and a new offer, the test is dead on arrival. Decide the success metric before the test starts, not after the results arrive. And size the slice to the decision: a 10% holdout protects revenue but takes months to say anything, a 50% split answers in weeks. For a first test, I would rather have a fast answer on a small workflow than a slow answer on the whole program.

Level three: say what you know and how well

The last layer is the reporting, and it decides whether the first two survive contact with a skeptical audience. My format is one line per claim, each carrying its evidence type and a confidence label:

• “Newsletter production takes 40% fewer hours, redeployed into a second weekly send” - baseline comparison, high confidence.
• “AI-assisted subject lines lifted opens 6% in a randomized half-list test over four sends” - holdout, high confidence.
• “Organic traffic rose while AI content shipped” - correlation only, low confidence, not claimed as caused.

The third line matters most. Including a correlation labelled as one buys credibility for the two lines above it, where excluding it would just invite someone else to bring it up. This is the same discipline as reporting an ROI figure with its confidence attached: the report that admits what it does not know is the one that gets believed next quarter too. I have watched precise-looking numbers lose that trust exactly once and never win it back.

What I refuse to do in these reports is let a model fill the gaps. An AI asked to estimate the value of its own adoption produces the vendor calculator’s answer with more words around it. Numbers come from the baseline sheet and the test results. The model can help write the narrative.

A 90-day proof plan

If the request for proof has already landed on your desk, this is the sequence I would run, and none of it requires a developer or an analyst:

1. Week 1: pick the two workflows where AI is most used. Reconstruct their before-baselines from old timesheets, calendars or the person who did the work. Imperfect reconstruction beats none - write down that it is reconstructed.
2. Weeks 2-5: run the list-split test on email, because it is the fastest clean result available. Four sends is enough for a direction.
3. Weeks 2-13: run the content cohort in parallel, since it needs a quarter to say anything about performance.
4. Week 13: report all three levels in the one-line-per-claim format. Take the strongest verified claim, not the largest number, as the headline.

The first report from this plan is modest - deliberately so, and what a realistic first quarter looks like month by month is worth agreeing with stakeholders before the clock starts. It is also the first one in most AI programs that contains a tested number, and in my experience that changes the conversation from “prove it” to “test the next thing” - which is where you want the conversation to live.

FAQ

Isn’t incrementality testing only for big advertisers? The technique scales down further than the vendors selling it suggest. A randomized email split is an incrementality test. So is holding five posts out of the new pipeline. What big advertisers have is more traffic, which buys faster significance - a small team just tests bigger differences for longer.

What if the holdout shows AI made no difference? Then you learned it on a slice instead of the whole program, which is the point. A null result on one workflow is not a verdict on AI - it usually means the workflow’s bottleneck was never the part AI accelerated. Kill or fix that use case and test the next one.

How do I handle the AI influence I really can’t measure, like being cited by assistants? Track it at the aggregate level - branded search volume, assistant referrals in GA4, a “how did you hear about us” field - and report it as directional. The unmeasurable part of AI’s impact is real. The mistake is dressing it in a precise number.

This post is part of the analytics and ROI guides. The workflow-level formula behind level one is in how to calculate AI marketing ROI, the attribution collapse behind all of it is in marketing attribution in the zero-click era, and the wider map is the complete guide to AI marketing.