The traffic impact of AI Overviews
An analysis of 1,675 keywords shows AIOs could reduce organic clicks
A warm welcome to 115 new Growth Memo readers who joined us since last week! Join the ranks of Amazon, Microsoft, Google and 12,500 other Growth Memo readers:
Brand-centric, and Context-aware, TheWordsmith.ai redefines AI content creation
If you are tired of AI content tools being just trigger-happy to start spitting out genetic content, then TheWordsmith.ai is worth a look.
Your company and brand DNA is the foundation of all content created
We tap into real-time external data sources to supplement the LLMs so that the content is timely and relevant to your audience
Get fine-grained control over the content creation, whether you want to include first-party insights, or simply provide specific prompts or instructions
Privacy-first: No company/brand information, prompts or content ever get used to train LLMs. Your data remains yours.
Subscribers of Growth Memo get 50% OFF on any subscription, forever. Use promo code GrowthMemo50 at checkout. Try for FREE
Here is a quick overview of how full-length blog articles are created using TheWordsmith.ai. We are confident, you’ll never want any other AI tool to create content for you. Learn More
Two weeks after Google rolled out AI Overviews (AIOs), we can analyze early data to gauge the impact on organic traffic.
I wanted to understand the impact and implications of AIOs, especially after many reports of misinformation and sometimes harmful recommendations from Google's AI answers.
I found that AIOs can hurt organic traffic, especially when queries indicate that users want quick answers. However, there might be a chance that some AIOs deliver more traffic to cited sites.
When AIOs show up
I used ZipTie to crawl the search results for 1,675 queries in the health vertical on May 22nd to understand how, when and maybe why Google shows AIOs. I mixed ZipTie's data with Search Console and ahrefs to understand the implications for organic traffic, the influence of backlinks and estimated domain traffic.
42% of queries show AIOs
AIOs showed up 42% of the time (704/1,675 queries), which is much more than the 16% found in e-commerce by ZipTie1 or the 15% found by SEO Clarity.2
The higher rate makes sense since Googles announced to show AIOs for complex queries, which are more likely to occur in health than e-commerce. I found a weak relationship with the number of words, indicating that longer queries are more likely to trigger AIOs. Yet, I'm surprised to find so many AIOs in a sensitive space like health, where wrong information is so much more risky.
There seems to be no relationship between keyword difficulty or search volume, even though the domain I have GSC access to is more likely to be cited for low-difficulty queries.
AIOs are more likely to show up alongside People Also Ask (PAA), Featured Snippets (FS) and Discussions & Forums modules (D&F), which makes sense since those SERP Features indicate informational searches. Knowledge Panels and Top Ads showed no correlation but also showed up less often than other SERP Features.
Though correlations are weak, the data indicates that AIOs are more likely to appear for queries related to questions and comparisons.
Who shows up in AIOs
The 704 AIOs in the dataset cited 4,493 sites, which is an average of 6.3 links per AIO.
I found a very strong correlation (0.917) between sites that show up in the top 10 organic results and sites cited in AIOs for the 1,675 keywords. Surprisingly, Reddit (#92 on the list) and Quora (#17) barely contributed to citations.
I found no strong relationships between domains that get a lot of AIO citations and their organic traffic, backlinks (referring domains) or # ranking keywords.
Sites that rank well for keywords are more likely to be cited in AIOs, but it's still unclear when Google decides to cite a site that doesn't rank well for a keyword.
Traffic impact of AIOs
The most important question is how AIOs impact organic traffic of cited and uncited URLs. To get to the ground of things, I compared organic clicks from Search Console for a domain across 1,675 non-branded keywords (US) in week 5/7 with week 5/14. After excluding low-traffic keywords, 560 AIOs showed up for 1,344 keywords, of which the target domain was cited 171 times with 461 different URLs. To make sure rank changes don't influence the results, I excluded keywords with a higher rank change than 0.9 and lower than -0.9, after which 52/521 URLs remained.
I found a strong negative correlation of -0.61 between cited URLs and traffic change, indicating that AIO citations send fewer clicks to cited URLs. In this case, the domain received -8.9% fewer clicks when cited in AIOs.
HOWEVER, results can vary by user intent. Most URLs that lost clicks, most likely due to AIOs, targeted questions like "how to get viagra without prescription". AIOs seem comparable to Featured Snippets, which can send more or fewer clicks based on whether the keyword is shallow or complex.
Also, note that a big chunk of the traffic losses was caused by a few URLs. Correlating the data without outlier URLs actually resulted in a slightly positive correlation of 0.1, indicating that there might be a chance that some AIOs send more traffic to URLs based on whether users want more information.
When AIOs show up and a site isn't cited, I found an average loss of -2.8% organic traffic, indicating that users might still click organic results. But to really make sure, we have to measure the impact on organic clicks for the same keywords. Take this result with a grain of salt.
Caveats
AIOs can change, and it's not clear how often. There does not seem to be a normal change rate over time, but it seems Google has pulled back a lot of AIOs. I wasn't able to recreate most AIOs five days after the initial crawl. Google might have pulled back due to the numerous reports of misleading and sometimes harmful answers.
1,675 queries are a good start, but we need 100 times as many to make more robust statements.
We also need a lot more traffic data than n=1.
Canary in the coal mine
In aggregate, strong SEO performance seems the best way to appear in AI Overviews, even though Google will also cite URLs that don’t perform well. We still don’t know enough about the content features that make it more likely to be cited in AIOs.
Broadly speaking, AI overviews have a bigger impact than "just" Google Search. They’re the first AI feature that significantly changes the cash cow of one of the most valuable companies in the world. AI has been undergoing a massive hype cycle since the launch of Chat GPT3 in November 2022. Lately, we've been asking ourselves more and more how big the actual incremental value of AI is. AIOs are the canary in the coal mine. They could prove the value of AI in products or even pop the AI bubble we're in.
Great analysis dude. And perfectly timed as I’m presenting on this on Wednesday - I’ll definitely be citing you!
I wrote about this article of yours, which I liked:
https://estilo.substack.com/p/ai-overviews-google-esta-jugando
After, I posted a note: https://substack.com/@leyderecho/note/c-57481889