Personalization Guide
The 2020 Edition

The Marketer's Guide to Website Personalization

Save for later
Chapter 6:

Measure personalization performance

Now that you understand what’s possible with personalization, and what’s necessary to deploy it — you’re on your way to developing the B2B marketer’s personalization philosophy.

What exactly is that? It’s when you start to realize that more individual web pages create better online experiences for your visitors and that when you start catering to an individual's unique needs.

And crazy enough — you see results.

To start effectively personalizing your website, you first need to get into your head what is possible with personalization. Think about how your site functions today.

Are there points where you think you could collect data more clearly?

Do you build your pages with one person in-mind or are they built for a wide audience? Can you surprise and delight your visitors more frequently than you current are?

Try out some of the methods mentioned in the previous chapters — and you’ll be well on your way to delivering more enjoyable and higher-converting experiences.

If you start to think about everything you do with a two-fold approach, you’ll quickly start to realize you have two main goals as a B2B maketer:

  1. Collect as much data as possible on your visitors.
  2. Identify every point where you can make an element on-page more personal for your traffic.

Personalization vs A/B Testing

Just like the terms customization and personalization often get conflated by marketers, so do personalization and A/B testing (also referred to as split or multivariate testing).

Let’s get one thing clear: these concepts are not the same thing. But they can and often do, work hand-in-hand.

Example of a CTA A/B test on an e-commerce page

A/B testing is a method of randomly displaying two or more versions of a web page to a random audience to see what version of a page performs better. You change up some elements on a page, and then see the effects on the performance of the page (such as conversion rate, email submissions, CTR, etc).

The goal is to improve the site for everyone and build a better performing page. It’s a global experiment.

With personalization, you are defining your audiences explicitly. You’re coming up with a hypothesis that by making elements on a page more relevant to specific groups of people, you’ll see an improvement in performance. It follows a similar same thought process as an A/B test, as you’re making a rationale hypothesis and deploying an experiment. The difference is that the personalized experience shouldn’t necessarily be designed for all of your traffic.

The personalized experience you launch might be designed solely for SaaS or E-commerce visitors. And it might be perfectly optimized for churned customers or trial registrants.

You can still run an A/B test in congruence with a personalization, but you’d likely see it most frequently run in two ways:

1) A/B test personalization vs no personalization on your site

In this scenario, you’re wanting to discover if personalization helps you reach your target metrics. Most customers we’ve worked with have done an initial test of this sort by setting up an A/B test with software such as Google Analytics, VWO, or Optimizely.

To test the effect of personalization, they will establish a holdback (a percentage that doesn’t see the personalization experiment) and test it against a personalized version of their website.

The higher the percentage you make the holdback (i.e. 50%), the faster you can reach significance on your test and hopefully see the power of personalization.

2) A/B test one personalized experience vs another personalized experience

This is the most common way you’ll see A/B testing and personalization used in unison. In this scenario, you’ll again establish a holdback (slightly less than before, let’s say 10%) and then deploy several personalized variants to test. You’ll deploy them all to the same audience bucket, and you’ll see what type of personalization performs best for that particular audience.

By doing a test in this way, you can continuously improve the effectiveness of on-site personalization. We recommend A/B testing only your largest audiences because reaching significance (to a 95% threshold) can take a bit of time if your traffic is low or your audience is too segmented.

And it should be noted that these tests shouldn’t last forever. Data driven marketers see success from consistently formulating new hypotheses and trying them on a site.

When should you start personalizing your site?

  1. All of this information is great and all, but will it actually work for me?
  2. Is it too burdensome to implement on my site?
  3. Will marginal increases even matter at my company’s size?
  4. Will the ROI be positive for my brand?

We hear you — and when we started personalizing our own site, these same questions flowed through our minds.

When should you start website personalization?

After conducting a market analysis and working with our first cohort of Proof Experiences customers, we’ve identified a few indicators that signal that you could be ready to personalize:

  • You’ve reached Product Market Fit
  • You’re using A/B testing, analytics, and data integration tools
  • You get 10,000 or more visitors per month to your site
  • You have a marketer who can manage personalization experiments along with normal conversion rate optimization.

Does this sound like you? Sign up for a demo of Proof Experiences!

How do you measure success?

When starting a personalization program at your business, we recommend identifying a KPI that you want to focus on improving in the long-run through your experimentation. In most cases, our customers choose this KPI to be revenue-focused metric such as New MRR or Customer LTV.

These are great metrics to measure, but since they are down-funnel and take time to adjust from top of funnel experimentation, we also rely on tracking metrics that are more measurable in the short-term. Since we’re deploying our experiments on marketing pages, we want to help our customers increase their customer acquisition efforts. The most practical way to do this is to look up funnel — ideally towards a cohort’s action towards a next step. What do we mean?

Let’s consider an example — imagine you have a homepage with one CTA pushing your traffic to signup for a demo. We would deploy an experiment to help increase click through from that button to the next page. In that instance, we would accomplish that deployment by building custom buttons for each audience.

Then, when tracking, we’d see if these more tailored experiences increased click through and demo registrations, as both of these are leading indicators to higher revenue.

You’ll continue watching your experiments and monitoring down-funnel revenue KPIs, and after reaching significant data, you can wrap up your experiment. We usually are able to reach results with about two-weeks of data, and we try to wrap up experiments within a month.

Then we go back to the drawing board and think of other tests to run.  This high-tempo testing mindset helps you continuously improve.

It’s important to note that the reason we can quickly and deliberately run experiments is that we’re maniacal about tracking. We Amplitude to set up dashboards around everything we are testing on site — from traditional A/B tests to personalization experiments.

Here’s how to setup dashboards to monitor your personalization performance.

The team you need to personalize

On a practical perspective, you likely want to know — “do I have the skills to run this or do I need to hire someone?”

If this guide felt applicable to your business and most of the concepts weren't foreign to you, we think you’ll be able to quickly launch personalization with Proof Experiences. And if you’re a marketer or has a marketer on your team that has dealt with Experimentation before and tools like Google Optimize, Segment, Heap, Optimizely, Segment, and others don’t scare you— even better!

We’ve found that growth marketers, CRO analysts, marketing managers, VPs of marketing, and other data-driven roles pick up these concepts quickly.

In other situations where you don’t have the time or skills to dedicate to managing Experiments internally, we are able to provide more outsourced management of Proof Experiences. Contact our team to find out more.

You've made it through The Marketer's Guide to Website Personalization! Next step? Share the summary with team to get the ball rolling. Be the champion in your company to hit your growth goals.

right arrow