(R)evolution
On May 10th, 2023, Google opened the doors to its most ambitious change to Search: SGE. The question is, “Is it an evolution or a revolution”?
Evolutions are marked by gradual and incremental adaptation to a changing environment. Revolutions stand for disruption and fundamental shifts. Is SGE the natural next step in Search or a new game with new rules? Based on the answer, dominating players either solidify their position or lose their advantage and have to start from scratch.
8 months after launch, it’s time to take inventory and see how SGE has changed. We don’t know when SGE will launch. My theory: Google keeps SGE on the ready like a reservist in case other players make significant moves, but it’d rather not push its cash cow over. Recently, Ross Hudgens noticed Google uses language for SGE that indicates it might stay in beta.1
SGE is continuously evolving within its beta sandbox, with new analyses shedding light on its changes and offering insights into whether it represents a revolution or an evolution in Search.
11 SGE observations - revised
In May, I wrote a Memo about the 11 realizations I had when testing SGE for the first time. Half the realizations have already changed:
1/ Google doesn’t show AI results by default.
👎🏻That changed: Google shows SGE for almost every query but varies between a pre-populated result and opt-in.
2/ The cost argument is not as big as many think.
👍🏻SGE shows up for almost every query, validating that cost is not as much of an issue as it’s made out to be. At least 10 million people are in the SGE beta, and we haven’t noticed a spike in Aaphabet’s CaPex (capital expenditures) over the last quarters.2
This month, Sam Altman, the CEO of OpenAI, addressed this question while being a guest on the Bill Gates podcast. He said the costs of running their earliest version of GPT, GPT3.0, in 3 years came down by a factor of 40x. The costs of running GPT3.5, which has been running for just a little over a year, came down by a factor of 10x. To add to it, he said that the costs of running LLM models are on the steepest slope of cost reductions that he had ever seen, much steeper than Moore's Law.3
3/ We can reverse-engineer AI results.
👍🏻Based on the studies we've seen lately, it seems more and more likely that we can reverse engineer SGE to a meaningful degree. I expect a lot more to come.
4/ By explicitly writing about an angle AI Snapshots highlight, websites might increase the chance of ranking in the carousel.
👎🏻This hasn't been tested enough for me to say it still holds.
5/ AI answers in e-commerce are the most aggressive.
👍🏻That still holds very true and has become a more important point than I realized at the time (more below).
6/ Product, local and brand reviews are gaining significant importance for AI results.
👍🏻Also yes, Google heavily leans on 3rd and 1st party reviews.
7/ Google might offset higher costs of generative AI queries with more ads in the search results.
👎🏻No signs of ads in SGE itself yet, but there is a lot of surface. E-commerce product listings, for example, could easily feature sponsored products.
8/ Sites that provide local reviews might actually get more traffic from Google based on the SGE beta.
👍🏻Still holds true: SGE adds no value added in local search.
9/ AI answers for YMYL topics are very uncertain.
👎🏻To my surprise, Google has amped up the frequency for healthcare while keeping it low for finance-related queries.
10/ It's a lot harder for Google to fine-tune AI answers than classic search results.
👎🏻The opposite seems to be the case, especially with new engines like Gemini behind SGE.
11/ The biggest opportunity for SGE is not in Google Search but in becoming an assistant for Google’s whole ecosystem.
👍🏻The signs point in the right direction. Microsoft's launch of Copilot on the web and natively in Windows. Bard attempted to search through Google Drive files and emails as well (not accurately so far).
The ecosystem is adapting as well: since the launch of Chat GPT, 88% of news sites in the US have blocked AI crawlers.4 Allow me a moment of humble bragging: I predicted 84% in Should you block GPTbot.
Based on my analysis, the share of top 1,000 sites blocking Chat GPT is likely to jump up to 84%.
Extracting trends from 3 SGE studies
To better understand how SGE changed since its inception, I'm cross-referencing 3 SGE analyses:
Authoritas looked at 1,000 commercial queries, desktop, December ‘237
Brightedge looked at 1 billion queries across 9 industries8
This is what they found out:
Classic ranking factors might not apply to SGE
❗Shocker: most of the time, SGE doesn’t pick content from pages ranking in the top 10 results for its answers:
Onely found that 43% of the time, SGE goes beyond the top 10 to source answers.
Authoritas found that only 4.5% of SGE sources came from the top 10.
Two of the three studies came to the conclusion that ranking at the top in classic organic Search doesn’t automatically lead to being the top cited source in SGE.
As a result, it’s likely that SGE works fundamentally differently than classic Search. While the crawlability of your site still matters, factors like duplicate content seem to not play a role. As a result, structures like paginations might be more obstacles than helpful.
It’s unclear whether Google intentionally ranks other results in SGE to diversify Search more or if there is an important difference between the content. The domains that ranked most often in SGE in the Authoritas study (thanks for making the raw data public 👏🏻) are:
Wikipedia
Yelp
Google
Investopedia
Mayoclinic
Forbes
However, when we look at the ratio of classic organic search rankings and citations in SGE, we see a mix of very and less popular domains:
Those domains would get more visibility if SGE rolled out today. However, keep in mind that the Authoritas study is based on a small sample of short-head keywords, and we don’t know how many users would click on citations in different types of SGE modules.
I found an interesting relationship between citations in SGE and PAA (People Also Ask) SERP Features in the Authoritas data. With a coefficient of .77, the correlation is very strong.
Note that the correlation has a Pearson coefficient of .53 without Wikipedia, which is a strong outlier. It’s still strong, but not as strong as with Wikipedia.
Another interesting pattern is which site types make up most SGE citations from the 1,000 keywords Authoritas analyzed: we find local and dictionary sites at the top with a big gap to other types.
Especially publishers and affiliates are not cited as often as I would have expected. Deeper analysis is needed to understand the reason.
SGE everywhere
The number of queries that trigger SGE has grown:
Onely found that SGE appears for 78% of queries, of which 47% show a pre-populated AI snapshot and 31% are opt-in (“generate” button).
Authoritas found similar results: SGE shows up for 86.8% of queries. The opt-in shows up 34.1% of the time, while users get a pre-populated AI snapshot 65.9% of the time.
Battlefield: e-commerce
Answers in SGE have gotten longer: from 3 paragraphs in June to 5 in December. They feature 10.2 links from 4 unique domains on average, which indicates that SGE corroborates answers from several passages on the same page. It seems that the number of sources has grown from ~3 to ~5 in June to ~7 in December. As SGE gets smarter, it pulls from more content sources, which hopefully means sending traffic to more domains.
A strong commonality between all studies is that healthcare shows SGE a lot more often than finance. While I expected Google to shy away from health to not risk giving false advice due to hallucination, it seems that SGE comes too close to giving financial advice, which is illegal unless you’re certified. The lines are blurry: providing medical advice without a license is also not allowed.
For verticals, two studies found directionally overlapping results except for e-commerce:
Onely: E-commerce: 92%, Brightedge: 49%
Onely: Healthcare: 87%, Bridgtedge: 76%
Onely: Finance: 24%, Brightedge: 17%
Part of the reason could be that the SGE results vary so much in e-commerce. A new type of snapshot shows variations of the same product at different price points.
Integrated e-commerce carousels launched around November 9th
The push in e-commerce seems no accident: Google has been very aggressive in turning its search engine into a marketplace. I attribute the speed of progression for SGE in e-commerce to the growing competition with Amazon and other retail marketplaces.
From e-commerce shifts:
Google’s metamorphosis into a shopping marketplace is complete. Two ingredients were missing: product filters that turn pure search pages into e-commerce search pages and direct checkout. Those ingredients have now been added, and the cake has been baked.
Instead of product listings, SGE comes much closer to a shopping assistant who helps you narrow the list of products down and compare price points.
Bottom line
We’re starting to uncover important insights into the works of SGE. If early indicators are true, the mechanics behind citations could differ vastly from classic Google ranking factors, even though modern search LLMs ground models in classic search ranks.
Google keeps iterating on SGE. But let’s not forget that the environment around Google also changes. Users might search with much longer queries now that LLMs allow them to search in natural language. Most studies look at longtail queries, but do they still matter for AI Search?
Based on our observations so far, we can say with high certainty that Google verticalizes the SERPs for SGE just as it does for classic Search. As soon as you use a query modifier that indicates that you’re interested in shopping or local results, for example, Google shows specific SERP Features. SGE adapts in the same way.
Switching into personal opinion and crystalballing mode, I can see 3 scenarios for SGE moving forward:
First, SGE breaks into smaller features that Google tests in the SERPs instead of launching the full version we’re seeing in beta today. One example is integrated carousels (see screenshot above) that use AI in combination with the shopping, knowledge or topic graph and augment the experience with price comparisons, explanations, and context.
Second, Google personalizes the SGE experience to build stronger network effects. Google could show you different answers in AI snapshots based on your shopping preferences, the publishers you subscribe to in Google Discover, or the channels you subscribe to on YouTube.
The more Google sees its business model threatened, the more it has to capitalize on the Platform Confluence, meaning bringing products closer together.
I also don’t understand why Bard and SGE are separated. In my opinion, there is a chance they will merge over time and break out of search and into Gmail, Chrome (like the 3 new features Google recently announced9), and YouTube.
Third, Google launches a marketplace for SGE/Bard plugins. Just like OpenAI launched a plugin marketplace for Chat GPT, users will be able to activate plugins for SGE to personalize and improve their search experience. Imagine Tripadvisor answering travel or G2 software questions.
Viva la revolucion!
Super insightful, thanks Kevin! I am actually finding SGE quite useful as a user but I still make lots of clickthroughs. Very curious to see how it evolves.
When you say SGE is correlated with People Also Ask, what do you mean? They are both showing up for a lot of the same keywords?
I like that you came back to the observations. :)
Regarding your outlook on the 3 possibilites: I think #1 is the most likely outcome. I think before they add anything like #2 to SGE, it seems more likely they would try to come up with something like that in regular search. A product they know that works and is standing on a shaky foundation at the moment.
P.S. You got one thing mixed up from the Authoritas study.
They state that for their keyword set the "generate_button" appears for 65.9% of keywords and the auto-generated/pre-populated "show_more" takes the remaining 34.1%. In the article you say it's the other way around.