Content Goblins
Shot: On November 1st, The Verge published 8,855 words about SEOs ruining the internet.
Chaser: Hours later, Google’s Search Liaison Danny Sullivan responded with 4,636 words.
Other publications in the SEO space followed. The Verge team picked it back up on the Vergecast.
The common denominator between all of this? Outrage.
26 old me would have written an emotional critique of The Verge’s article because it triggered imposter syndrome. 36 old me still feels waves of imposter syndrome but can pause and reflect for a moment. All 3 players - SEOs, publishers and Google - have a role to play in the decay of the web.
Trigger warning: no one gets a free pass in this Memo, not even me.
SEO - content goblins
SEOs, referred to as “content goblin willing to eschew rules, morals, and good taste in exchange for eyeballs and mountains of cash” in the narrative journalism article by The Verge, have an outsized impact on the quality of the web.
Most people still see SEO as spam. The comments under Nilay Patel’s (chief editor at The Verge) Thread show a lot of sympathy for the author and negative sentiment against SEO.
Quick anecdote: Tobi Lütke, Founder and CEO of Shopify, once Tweeted that lots of SEO recommendations are snake oil. When we, the SEO team at Shopify, asked him about it, he clarified SEO consultants are snake oil salesmen. I felt much better about that answer back when I was still at Shopify than today. But the point is that SEO doesn’t have a good reputation.
I get where the notion comes from. When you’re reading this, I’ll be on a plane from Thailand to the US, returning from the excellent Chiang Mai SEO (CMSEO) conference. Many people at CMSEO stress test Google’s systems to the max and learn a lot in the process. Some pump tens of thousands of domains with AI-generated content into Google’s index with the purpose of generating backlinks. The ones I talked to are genuinely curious to peel back the layers of the Google onion, and none of them wants to trick people into money schemes. But SEO spammers exist, and stress testing Google’s systems can pollute the web.
I dunno about goblins, but humans tend to exploit systems. Every industry has spammers, charlatans, schemes, and unfair monopolies. The FTC reported over $8b in fraud damage in 2022, +30% more compared to 2021. It’s a fallacy to think that SEO is a specific group of people; the web would be better if it didn’t exist, and SEO never became a thing. (source)
The SEO community is divided into white hat (play within the rules) and black hat (play beyond the rules). White hat SEOs work at companies, agencies or make a living from their own web projects. None of them can risk short term gains for significant long term damage. Many black hat SEOs who have played the game for a while are tired of constantly worrying and buying legit businesses.
I argue that some of the spam created by black hat SEOs doesn’t pollute the web as much as mediocre content created by white hats. When Google finds black hat schemes, it takes the whole construct down. Mediocre content on reputable domains, however, might lose some visibility during algorithm updates but generally doesn’t cause harm. It just also doesn’t create a lot of value.
I’ll be the first to admit that I’ve prioritized search engines over users and traffic over everything, especially early on in my career.
⛏️The underlying problem: Marketing has become too quant. Most teams look only at search volume but not customer research anymore. User research has been replaced by performance marketing and now lives with UX or product teams. The result is content that reads robotic and dull, sometimes trying to be cringe-worthy funny.
🔐 The unlock: Marketers should go back to using qualitative customer insights to tailor topics, wording and pain points to their target audience. AI provides scalable ways to do that: we can query call transcripts or data sets at scale and turn them into precious information with tools like Humata AI or Chat GPT plugins. LLMs allow us to fish out valuable insights from conversations.
It works. I’ve used insights from conversations in content for a former fintech unicorn client and strongly grew pipeline, conversion rates and user engagement as a result of higher quality content.
Publishers - goblin traders
Publishers are the largest source of content on the web. But the industry is falling apart.
Advertising, the biggest source of publisher revenue, moved from publishers to advertising platforms like Google or Meta. From 2008 to 2018, US publisher revenue nosedived from $37.8b to $14.3b (-62%), while digital advertising reached $107b. (source)
As a result, publishers shopped for new business models: affiliate marketing, AdSense, subscriptions, or renting out subdomains. All of these models are funded by SEO traffic, whether it’s Top Stories, Google News, Google Discover, or classic organic search results.
My theory: Google prefers affiliate content from publishers over pure affiliates because they have editorial guidelines.
SEO has given life to companies that cannot afford to pay for traffic. The current economic environment forces a lot of companies to drastically reduce paid spend and make up for it with organic traffic. ZIRP (Zero Interest Rate Policy) times allowed companies with strong funding to buy market share. An advertising report by Zenith shows the global advertising market was expected to grow 11.2% in 2021 to total $669 billion, $40 billion more than what was spent before the pandemic in 2019. But a year later, after going from 0% to 4% FED funds interest rates, ad spend forecasts were lowered and even more so for 2023. Now, buying market share is barely possible because money is too expensive and big funding rounds have slowed down. (source)
10 years ago, organic traffic from Google accounted for ~66% of publisher traffic. Since then, social networks have significantly increased and then reduced the visibility of news, making Google the last viable channel for publishers. (source)
⛏️The underlying problem: Most young people (Gen Z) get their news from TikTok, which hurts Google and the AdSense revenue for publishers.
Audiences pay more attention to celebrities, influencers, and social media personalities than journalists on platforms such as TikTok, Instagram, and Snapchat.
[…]
The Chinese-owned social network reaches 44% of 18–24s across markets and 20% for news. (source)
Editorial guidelines and standards at classic publishers don’t always grip, but at least they have some. TikTok’s content is user-generated and doesn’t go through a fact-checking and review process.
In his testimony as part of the current DOJ antitrust lawsuit against Google, Prabhakar Raghavan, who runs Search and Ads at Google, didn’t miss the opportunity to point out - once again - that young people prefer searching on TikTok over Google. This might be a Red Herring since TikTok’s growth has slowed down quite a bit, and Google needs strong adversaries to avoid the impression of being a monopoly. (source)
🔐 The unlock: There is no path to bring publishers back to former glory. Just like the internet unbundled the publisher business models in the late 2000s, journalists and writers unbundle from newspapers by going directly to customers via newsletters or YouTube channels.
The publisher market consolidates:
A few big ones capture most of the market (NYT, WSJ, etc.)
Some build business models by specializing in business (The Information, The Economist)
Others find innovative business models (Axios)
Many journalists go independent and direct-to-consumer
Google - the goblin king
SEOs and publishers try to please the goblin king, Google, in the hope of getting a piece of his treasure.
If the people are mad at spam goblins for destroying the web, they’ll be furious very soon. As you read this, spammers are using AI to 10x their output. There is a real chance Google might not be able to handle it without drastic measures. Even further, AI content threatens the whole business model of Organic Search.
3 years ago, I made the argument for indexing APIs. Since then, the adoption of Bing’s IndexNow has grown, but Google hasn’t moved an inch. I think it’s time to reconsider. Google, fighting a trench war with spammers, saw a 5x increase in detected web spam from 2021 to 2022. SpamBrain detects and squashes spam before it gets indexed, but now that AI spam content can be created within seconds, indexing seems to be the best bottleneck to catch it.
One of the best ways to get ahold of AI spam in Search is flipping the playbook on its head and asking webmasters to bring content to Google, instead of the other way around.
Filtering out low-quality content and spam is just one part of the job of making the web better. Replacing spam with good content is the other, and that’s where it gets hairy. A lot of content creation has moved to apps like YouTube or TikTok, contributing to less valuable content on the open web for Google Search.
⛏️The underlying problem: These days, I cringe when people separate “content for Google” from “content for users”. And yet, not everything that’s good for readers is good for SEO. The winning playbook in SEO often leads to mediocre content, but it works. The harsh reality is Google hasn’t figured out how to reward the best content and filter out fluff. To some degree, you need to blame the game, not just the players.
Google’s rules for Search have become so arbitrary that the boundaries of the game are hard to see. “Just create good content for people instead of search engines” is aspirational at best. And even when playing by the rules, Google’s algorithm updates have become hardcore. (source)
Nilay Patel has a point when he says, “It is just a little funny that his position is that Google publishes hundreds of pages of guidance but the best guidance is not to pay any attention to it.”
🔐 The unlock: Google needs to set clearer rules, improve its systems to truly reward good content, and reduce the ad load to make SEO traffic more attractive. I highly doubt it will happen.
Systems get better when they’re stressed in a reasonable manner. Google had no competition over the last 25 years. Who else pushed it to get better other than SEOs? The internet needs more SEOs, not fewer.
More ads in the search results make SEO less attractive, leading to lower investments in good content. Aaron Wall from SEO Book, fighting the good fight, has a strong point when he writes, “The great ROI days of SEO are for the most part over due to the heavy ad load of the SERPs.” Google isn’t able to turn the growing ad load around. It’s incentivized to keep growing and satisfy shareholder expectations. (source)
One question I get from my clients all the time is “what content strategy makes sense in times of AI?”
My number one answer: Make readers feel something.
The Verge piece certainly accomplished that, even though it missed a few important details.