Perhaps I'm suffering from a lack of oxygen/creativity on this flight so bear with me. How would you do an incrementality test with AI search/GEO/SEO. Programmatic pages for only some geographies? I'm struggling to think about how you'd build a MMM or incrementality tests with GEO.
This is very important for many executives to read.
We don't kill attribution, or make it irrelevant. We need to understand it's limitations and keep critical thinking alive, bets, and gut feelings to keep difficult or impossible to measure actions in the budget.
If not, we will be forced to keep the most expensive but trackable ways on the budget line items.
I have been advocating about this for a long time, but this is so well written and carries the authority I need to be using it from now on going forward :)
@Kevin, I would need this to be my battle-card item for high-level conversations!
Say you have a company that's present in a couple of countries. Then, you'd roll out a change in only one country on pages that also exist in the other country, wait, then revert. That would be a geo incrementality test to me.
Attribution failing for non-branded channels is worth naming, and the useful question is what replaces it rather than whether to keep it.
The substitute that holds up is a dated ledger plus one outcome the business already owns. A frozen prompt set per buyer stage, written once and not edited, because an edited prompt measures a different thing. Per engine, a row per run holding the model version, the date and the locale, since citation movement clusters around model releases, not your publishing calendar.
Then pair the proxy with a number someone else already answers for: qualified enquiries, pipeline, retention. Citation share alone cannot be reported as demand, and the pairing is what stops it becoming another dashboard nobody trusts.
Two guards. A model update is an annotation on the chart, not a win. And absence is stable while presence is not, so run each prompt more than once before calling a movement real.
Are your citation shifts clustering on release dates too?
The brand search point is sharp. Can't measure AI search directly, so you measure the shadow it casts. Stealing that.
Perhaps I'm suffering from a lack of oxygen/creativity on this flight so bear with me. How would you do an incrementality test with AI search/GEO/SEO. Programmatic pages for only some geographies? I'm struggling to think about how you'd build a MMM or incrementality tests with GEO.
This is very important for many executives to read.
We don't kill attribution, or make it irrelevant. We need to understand it's limitations and keep critical thinking alive, bets, and gut feelings to keep difficult or impossible to measure actions in the budget.
If not, we will be forced to keep the most expensive but trackable ways on the budget line items.
I have been advocating about this for a long time, but this is so well written and carries the authority I need to be using it from now on going forward :)
@Kevin, I would need this to be my battle-card item for high-level conversations!
Oh, thank you! Keep fighting the good fight ;-)
Say you have a company that's present in a couple of countries. Then, you'd roll out a change in only one country on pages that also exist in the other country, wait, then revert. That would be a geo incrementality test to me.
Attribution failing for non-branded channels is worth naming, and the useful question is what replaces it rather than whether to keep it.
The substitute that holds up is a dated ledger plus one outcome the business already owns. A frozen prompt set per buyer stage, written once and not edited, because an edited prompt measures a different thing. Per engine, a row per run holding the model version, the date and the locale, since citation movement clusters around model releases, not your publishing calendar.
Then pair the proxy with a number someone else already answers for: qualified enquiries, pipeline, retention. Citation share alone cannot be reported as demand, and the pairing is what stops it becoming another dashboard nobody trusts.
Two guards. A model update is an annotation on the chart, not a win. And absence is stable while presence is not, so run each prompt more than once before calling a movement real.
Are your citation shifts clustering on release dates too?