Here is how pros do conversion rate optimization
Here is how pros do conversion rate optimization
Insights from Notion, Graphite and ShipBob
Conversion optimization is not just complementary to SEO. It’s a quintessential part of the success of SEO.
Google’s goal is to answer the user's question. Sometimes, answers come in the form of buying or signing up for a product. Conversion Optimization, or CRO, is the craft of completing user journeys. So, the easier we make it for SEO traffic to convert, the more Google might send us.
What excites me so much about CRO is that it sits between marketing and product. It has a strong product DNA because it’s fundamentally about (website) features and user onboarding. We often forget that the website is a product and how many new features we can add or improve.
Many think CRO is about clever popups and CTA messaging. But it goes so much further. In this guide, which I wrote with some of the best conversion experts in the world, I show you how to leverage Interactives, think about Surfaces and Levers and build a conversion testing system that slowly but surely iterates towards wins.
And now that chatbots are becoming so intelligent and product-led growth tears down the walls between website and product, a whole world of new opportunities and possibilities opens up.
This expert guide features input from:
Travis Bernard, VP CRO @ Graphite
Chris Nolan, former Head of Growth @ ShipBob
Joshua Kim, Growth Marketing Lead @ Notion
(Thank you all so much for making this piece truly special!)
First principles of conversion optimization
The goal of Conversion Optimization is to increase the number of sales or sign-ups by growing the rate of users who turn into customers. As Joshua Kim, Growth Marketing Lead @ Notion, describes it: “CRO, like most other growth channels, is all about controlling and maximizing the inputs within your control”.
Conversions can be:
Online store purchases
Form fills
Newsletter sign-ups
Product sign-ups
Downloads
Ad clicks
Newsletter CTA clicks
Video views
Notice how some conversions are hard, i.e., users pay for something. Other conversions are soft, i.e., users engage with something. We often forget that engagement is the precursor to conversion.
Successful Conversion Rate Optimization rests on three core principles:
Understand user intent, motivation and friction
Run experiments
Focus on business impact
Understanding what users are trying to accomplish (intent, like buy, evaluate, seek inspiration, solve a problem), what motivates them (price, features, value, status) and where they encounter friction is key to developing unique ideas instead of blindly copy/pasting them from blog articles.
Darius Contractor developed a helpful framework called “Psych”. [link] It looks at which elements on a page excite the user to sign up (e.g., trust signals, visuals, copy) and which ones demotivate them (e.g., long sign-up forms). The idea behind Psych is front-loading exciting elements and back-loading friction in the user journey. Every experience has a total Psych score, and the goal is to keep it positive. Users should encounter more exciting elements than friction.
Wealthfront’s homepage, for example, addresses concerns (1, market volatility), makes a promise (2, high APY), shows the product (3), and uses trust signals (4) to increase Psych. Visitors get excited because the homepage signals a lot of credibility and returns.
Only after users are psyched does Wealhtfront show them a 5-field form, of which only 2 are mandatory. When users encounter the form, they’re already bought in and willing to fill it out.
Another example of Psych is Thumbtack. On search pages, which are low-intent moments (browsing), they guide users through their filtering system with dedicated questions, like “how often do you need lawn care”, “any preferences for the clippings?” and “how big is the lawn?”.
Every answer gets users a little more psyched about the results. At the same time, it helps Thumbtack match searchers with the right service providers (segmentation).
“The most impactful tests will further increase intent or eliminate barriers at high intent moments (i.e.,, sign up flow, cart page, payments page, account management page) or increase intent or eliminate barriers at high traffic / low intent moments (i.e. blog or programmatic pages).”
(Travis, VP of CRO at Graphite)
Such ideas aren’t simply launched; they’re the result of a systematic process around ideation, design and testing.
Testing is part of the Scientific Method, an iterative way to learn and understand. As you will read throughout this guide, one of the key takeaways in this guide is building a perpetual system around CRO. A consistent testing program is critical to succeeding in CRO for 3 reasons:
Humans are bad at projecting outcomes in complex or uncertain environments
Our brains are subject to over 500 cognitive biases
Instead of fragmented ideas, testing allows us to build on top of previous experiments to approximate truths
Good CRO doesn’t only find new levers to lift conversion rates but moves a company’s North Star metric, the single most important number in the business. If this number goes up, the company is successful. Experiments need to be aligned with the North Star.
Doing CRO “well” in 2023, simply put, means being in touch with business metrics at the top and the bottom of your marketing funnel. Gone are the days where “more leads!” is the sole KPI tied to a CRO expert. The best folks in the business can speak to the impact of an experiment not just on lead volume, but also on lead quality and downstream impact to revenue. They are also deeply aware of where their website falls within the larger funnel, and take pride in their understanding of how their domain fits into the tech stack by partnering with data, growth, and operations teams.
(Christopher Nolan, former Head of Growth @ ShipBob)
Surfaces and levers of CRO
The most basic framework for how to think about Conversion Optimization is Surface and Levers. Surfaces are pages on a website, in checkout flows or logged-in experiences that allow conversions to be triggered. Levers are how these Surfaces can be changed to stimulate conversions.
Think about Surfaces as views or pages on a website:
Pricing page
Homepage
Landing pages
Product and category pages (e-commerce)
Blog
Case studies, customer stories, testimonials
Lead generation tools, like quizzes or calculators
Logins
Forms
Checkout
Demos and product tours
Chatbots
Levers are how we can modify Surfaces to grow conversion rates:
Price
CTAs
Messaging
Creative
Interactives
Third-party payments and logins
Trust signals
Personalization
You can quickly imagine how many combinations there are between Surfaces and Levers.
Price. In CRO, you want to constantly validate your organization’s pricing “bets” by testing different packages, plans and tiers against each other. Price testing is straightforward in B2B, where you can test monetization models like usage-based, value-based, service fees, subscription, freemium, tiers and trials. In e-commerce, you can play with single purchase vs. subscription, time sensitivity (order now), or amounts (“3 for 2”). In consumer products, you can make introductory offers (“$1 for the first four weeks”). Think about different ways to experience value and reward higher commitments with affordable price reductions.
While being one of the most complex levers of testing, price can also be one of the most impactful ones. Especially in the current environment, where customers have less buying power and take longer time to make decisions, testing different plans might be a good idea.
CTA. Good CTAs manage expectations, create trust and engage users.
Most people visualize popups when thinking about CRO. But CTAs are more than just buttons that jump in your face. CTAs come in all sizes and shapes with different timing, position, messaging or creative.
Good CTAs manage expectations by giving users a picture of what happens after they click:
“after submitting this form, you can expect…”
“X minutes to complete sign-up”
Trial length
Commitment (i.e. capture credit card upfront or not)
Marketing communications
etc.
Build trust through social proof or value proposition elements. Show how many customers a product has, reviews, testimonials, or return policies to remove uncertainty for users. Messages like “We store your information securely” or email opt-in checkboxes on the form can convince skeptical visitors, especially for expensive products.
Not every call to action has to lead visitors directly to a hard conversion like a product sign-up. It can also lead to soft conversions like email newsletter sign-ups or notifications.
Messaging. The art of copywriting is a skill in and of itself. Smart Organic Growth experts understand that the right message in headings and body copy can move customers to convert more than almost anything else.
Messaging brings clarity and drives action by:
Highlighting value propositions
Explaining the product
Alleviating concerns
Segmenting target users
Creative. We live in a visual world. Images capture attention and paint an image in the user’s mind.
Creative can come in the form of:
Product pictures
Customer pictures
Product or customer videos
The goal of using creative is to elicit emotional responses, like trust, curiosity or excitement.
Interactives. A new trend in web experience is providing users small widgets they can interact with to get a taste of what product engagement is like.
Onpage engagement gets users interested, grows buy-in and increases the chance users sign up for a product.
Third-party payments and logins. Once users decide to take action, removing friction in the shape of lengthy sign-up processes moves the needle. Google, Facebook, and Apple offer one-tap logins that authenticate users with their respective accounts on these platforms. Since every person on the plant has an account with at least one of these platforms, the sign-up process becomes frictionless. The faster you get people into the product, the faster they can experience its value.
Trust signals. Little pieces of information or images give users confidence in their decision by reducing risk:
testimonials
badges
customer logos
customer promises
return policy (e-commerce)
availability (e-commerce)
delivery (e-commerce)
Personalization. 10 years ago, personalization was all the rage but the hype dried out. Now, better tech stacks and AI allow companies to display individualized chatbot messages, copy, testimonials or CTAs.
Not every website offers the same Surfaces and Levers. A good framework for identifying suitable surfaces is Integrators vs aggregators.
Integrators can optimize:
pricing, landing, product tour and content pages
lead gen tools
forms
Aggregators can optimize:
product and category pages
blog
checkouts
Both have the homepage, chatbots and logins in their arsenal of surfaces.
Running effective experimentation programs
CRO programs are systems that, when run correctly, inevitably produce results. Like research programs, a team applying the Scientific Method in a systematic matter will sooner or later find effective levers.
“At Notion, CRO has been a meaningful program for driving top of funnel growth for our business. But under the hood, it’s taken significant investment in tooling, process, and a strategic roadmap to get there.”
Joshua Kim, Growth Marketing Lead @ Notion
CRO programs are critical because they lead to a constant flow of experiments. More shots on goal significantly increase the likelihood of finding a winner.
There are different testing methods:
A/b testing: testing two versions of a lever on the same surface
Multi-variate testing: testing several versions of a lever on the same surface
Holdouts: making changes to 90% of the same surface but leaving 10% unchanged to measure the incremental difference
Before/after: rolling out a change and measuring the lift (pseudo-testing)
Market testing: rolling out a change in only one market (country or product category) and measuring the change against other markets
Picking the right one depends on the information you want, how quickly you need feedback, what success looks like, your target audience and resources. But it mostly depends on what numbers you look at.
When defining experiments, it’s important to select the right success metric. It should reflect the action closest to the change you're testing and connect to the North Star metric of the business. If you’re testing a new CTA on a product page, for example, you want to measure CTA clicks.
The right success metrics stem from a robust hypothesis. We need a clear definition of the theory we want to prove right. For example:
Customers don’t understand how the product works
Customers don’t understand the value the product provides
Customers don’t understand why the problem to solve is important
Guardrail metrics ensure you’re not optimizing too far in one direction and sacrificing other parts of the business. In the context of the CTA example, a good guardrail metric would be average order size to avoid customers ordering faster but less. They also ensure the validity of the test since constant guardrails with changing success metrics indicate that the treatment led to the desired engagement. The question is in what direction the needle moves.
Effective CRO programs have several key artifacts. The Ideas log is a list of all testing ideas and running experiments. The test roadmap is an overview of planned experiments. The test doc contains the details of a test, like hypotheses, run-time, test design, and results. Together, they align organizations around testing and allow for ideas to build on each other.
A good testing system brings all stakeholders together on a regular basis. At Notion, the web optimization lead holds regular grooming meetings with the web engineering teams to review the backlog, scope experiments, and set sprint priorities.
A commonly used prioritization framework is ICE (Impact, Confidence, Effort), PIE (Potential, Importance, Ease) or PXL. [link]
However, any prioritization framework you pick should follow execution principles:
Prioritize big over small impact
Focus on high-intent surfaces first
Aim for the North Star
Take more shots on goal
Joshua Kim gives us practical questions to find out which Surfaces to start experimenting on:
What pages get the most traffic?
Where are most of our North Star conversions coming from?
What is the general flow of the website/visitor journey?
What are our largest traffic sources, and how do they perform differently?
Every experiment starts with an idea. Test ideas can come from observation, research, or ideation. Observation happens through competitor analysis, heat mapping, user tracking and funnel analysis. Research can be conducted through user interviews, surveys and feedback. Ideation results from looking at best practices, industry trends and brainstorming.
Effective testing programs are systems that slowly but surely iterate toward larger impact. Even if the first experiments fail, the right systems discover new levers sooner or later.
CRO best-in-class examples
Even though you want to be careful in mimicking what works for others (always test), best practices give help us understand where the bar is, what good looks like and what Surfaces and Levers to pay attention to.
Travis Bernard, VP CRO @ Graphite, shares two examples of successful conversion experiments across e-commerce and SaaS:
1/E-commerce blog
The client was generating a lot of low intent traffic to its blog, but conversion rate was low. Through user research, we discovered that many users were confused about how the product worked. We added a "how it works" bar to the top of the blog, and conversion rate increased by over 20%. It worked so well that the client rolled it out on other pages, and it worked well there too.
2/ SaaS Landing Page
This one is simple, but all we did was change the button CTA to include "free." Through user research, we discovered that users loved that they could try the product before buying. That value didn't appear on the landing pages, so we surfaced it and saw a nice increase in performance.
Chris Nolan, former Head of Growth @ ShipBob shares 6 examples across SaaS:
1/ Drift uses its own chatbot to sell its product - a chatbot. Since AI has massively improved, chatbots celebrate a comeback as CRO surfaces. They provide useful insights into how customers think and can answer critical questions faster than they browse.
2/ Segment uses visuals in a smart way: users see and understand the core value of the product within 5 seconds or less. The visual reflects the look of the dashboard, showing available integrations and social proof (“Glossier”).
3/ Shopify is a great example of managing expectations. Under their email capture, they make clear “what to expect” after submitting the form. This not only vastly increases their marketing list but serves as a symbol of trust/credibility to the user.
4/ ShipBob displays thoughtful and relevant in-line CTAs within high-traffic, content-heavy landing pages. They’re more effective than a desperate, out-of-context demo or signup CTA.
Christopher Nolan, former Head of Growth @ ShipBob:
While pop-ups may be perceived as a less-than-ideal user experience, CRO experts should not shy away from testing relevant pop-up offerings to new or returning visitors.
5/ Demio allows users to try the product. Visitors can interact with the product UI and understand its value without requiring an email or personal information.
6/ BigCommerce built a separate area (Surface) for enterprise offerings. Enterprise and self-serve funnels have different mechanics, which is why it makes sense to split them out.
6 CRO best practices from me:
1/ Ramp is another outstanding example of product demos. As I learned at Atlassian, users want to see the product before signing up!
2/ NY Times makes users an introductory offer to try the product, which can be a useful product demo workaround for publishers. Users still need to overcome a small amount of friction - just enough to segment users willing to pay for the product.
3/ Salesforce manages expectations in its lead form in more than 3 places. Shopify has a very consumer-like product with a low barrier to entry. Salesforce, on the other hand, is an enterprise product with high commitment from customers. They require a job title to segment suitable leads, even though each field introduces more friction in the form.
4/ Mixpanel provides users with a fast one-touch sign-up.
5/ Airbyte shows relevant testimonials at check-out.
6/ Atlassian shows the product on its Jira landing page. What sounds intuitive and given is not always the case. After a big data deep dive into the user journey at Atlassian, we learned that users really want to see the product user interface before signing up.
CRO Tools
Tools in CRO don’t just help run the experiment but also feed new ideas into the system and align collaborators around the most fruitful experiments.
Empowering teams with market research:
Testing infrastructure:
Aligning and collaborating:
Popups/CTAs:
Personalization:
Important: Tools like Segment can be used to track anonymous visitors and their engagement, and associate those engagements to an identified visitor at the point of their signup. This insight is powerful when justifying development or design efforts against a website product tour, as your internal teams and leadership can start to see the impact of engagement at the top of the funnel on downstream business metrics.
(Christopher Nolan, former Head of Growth @ ShipBob)
The key challenges in CRO
Every profession faces challenges. CRO is no exception.
In 2023, Organic Growth experts working on Conversion Optimization must solve 5 core challenges:
1/ Google Optimize is going away. Google decided to sunset Optimize and Optimize 360 by the end of September 2023 and instead integrate Google Analytics 4 with tools like AB Tasty, Optimizely or VWO.
“Optimize was great because it was free, and it allowed teams to build a culture of testing without large overhead. With it gone, there will be a battle between the dozens of testing tools to grab market share for the free users. My concern with the change is that it will be harder for small teams to do testing.”
(Travis Bernard, VP CRO @ Graphite)
2/ Conversion Optimization must meet certain statistical requirements. However, rigorous testing is not always sexy. Some tools predict results with a “Bayesian” approach to testing, but it takes experienced CROs to interpret them. The danger is for inexperienced people to get excited about seemingly good results only to be disappointed when moving the winner into production.
A safer approach is to ensure each test variance (including the control) gets at least 100 conversions before declaring a winner.
3/ Conversion Optimization needs cross-functional alignment to move things forward, but many organizations find themselves in silos between decision-makers and experts running experiments.
At Notion, the web experience team has explicit permission to move fast and does not need approval from all stakeholders, given a promise that winner experiments go through the common design polish phase if they roll out into production.
4/ Companies often adopt page templates without testing if they provide a lift.
Conversion experts should at least test through scroll tracking, link engagement and heat mapping.
5/ New privacy regulations and the death of cookies complicate data gathering for testing. A/b testing tools that run client-side heavily rely on 3rd party cookies. Testing cookies is part of marketing and, therefore, often rejected by users, which makes testing a lot harder.
New regulations like GDPR (General Data Protection Regulation) or CCPA (California Consumer Privacy Act) reduce testing feasibility and force Organic Growth experts to find workarounds or test on other Surfaces.
CRO sits between SEO and product
I consider CRO a critical part of SEO. Google’s main goal is to help users complete buyer journeys. In other words, the higher the success rate of users converting, the better it reflects on your results in organic search.
CRO and SEO also reinforce each other. Both disciplines work on user experience improvements and aim for higher website engagement. Keyword research can be used for search optimization and messaging. User intent plays a critical role in success. Low-intent pages can convert more users with targeted experiments, while high-intent pages can get more traffic with search optimization.
"SEO gets people to the site, and CRO gets them to take action. Investing in both areas can be beneficial."
(Travis Bernard, VP CRO @ Graphite)
The concept of CRO goes beyond websites. It applies wherever a conversion happens, like when users see snippets in organic/paid Google Search. I would argue title and snippet optimization is as much part of CRO as SEO.
Over the last two decades, the roles of SEO and CRO lived and grew in isolation. At the same time, we’re preaching to tear down silos in organizations. In engineering, we’re breaking monolithic applications apart into microservices. Most Growth and product organizations work in squads where members of different crafts come together to form a group pursuing the same goal. So, why are SEO and CRO still two different crafts?
In my opinion, it’s time to think about a new role that covers both: Organic Growth experts.
