Part 9: The Landing Page Optimization & A/B Testing Process

Today we’re going to dig into the most fun, most satisfying part of course – testing! Believe me, after you’ve run your first successful test you’ll be hooked.

Every day that you don’t have a test running is an opportunity missed.

Today’s Course Outline

The landing page optimization process consistes of 6 steps:

  1. Define your goals and success metrics
  2. Build your first landing page
  3. Drive traffic to your page
  4. Gather insight into page performance
  5. Create a test hypothesis
  6. A/B test your hypothesis

Having a process for Landing Page Optimization (LPO) is the difference between making informed testing choices vs. coming up with random ideas.

The diagram below shows the 6 steps of the LPO process, which includes the preparation stages where you set goals, build your page and start driving traffic to your page. This is followed by a constant feedback loop where insight is gathered to fuel A/B testing hypotheses (a statement of what to test).

the landing page optimization process

1. Define your goals and success metrics

Conversion expert Peep Laja from ConversionXL.com recommends a 4-pronged strategy for defining your goals and success metrics:

Landing page optimization business goals process

Step 1: Define your business objectives

Peep says that this is the answer to the question: “Why does your landing page exist?”

Example: A business objective for an online flower store is to “Increase our sales by receiving online orders for our bouquets.”

Step 2: Define your landing page goals

Goals come from your business objectives and are mostly strategic in nature. So if we were to continue with the business objective example of increasing our bouquet sales, we would have to:

  1. Do x – Add better product images
  2. Increase y – Increase clickthrough rates
  3. Reduce z – Reduce our shopping cart abandonment rate

Step 3: Define your key performance indicators (KPIs)

A KPI is a metric (numbers) that’s connected to your objectives.

“A key performance indicator (KPI) is a metric that helps you understand how you are doing against your objectives.”
– Avinash Kaushik

For example: Our flower store’s business objective is to sell bouquets. Our KPI could be number of bouquets sold online.

Step 4: Define your target metrics

For your KPIs to mean something for you, they need target metrics. Our flower store sold 57 bouquets last month. For our imaginary flower store, we can define a monthly target of 175 bouquets sold.

Using Peep’s framework, you insure that the work you will be doing on your landing pages is relevant to your business goals.

TAKE AWAY
Defining your goals will guide the strategy behind your choices of messaging and imagery on your landing page. Your success metrics will clarify what you’re trying to achieve with your optimization and a/b testing.

2. Build your first landing page

Once you have your goals defined, you can start building your first landing page.

We’ve already discussed how to create certain aspects of your content, especially for the written parts of your page, but how do you piece it all together and ensure that your landing page is laser focused on a single campaign objective?

To do this, we’re going to share a process for content collection, creation and construction.

The landing page creation process

The key to creating a focused landing page is to leverage a concept known as congruent page design.

Congruent landing page design

Congruent design is the idea of aligning every element on your page so that they work together to communicate your goal as a single collective voice.

Our process will follow these tasks:

  1. Write the goal of your landing page on a piece of paper
  2. Print out and cut out the pieces of the landing page elements diagram below (choose a click-through or lead gen page)
  3. Order the elements by importance to your campaign to create a communication hierarchy
  4. Now flip the bits of paper over and write copy for each element on the back (write a description for any graphic/video elements)
  5. Read out loud what you have written, in the order you have laid them out
  6. If any element doesn’t fully support your goal, you know it needs to be rewritten. If any element feels out of order, you know it needs to be moved.

Lead gen landing page elements cut-out

To help you construct your lead gen landing page, print out the following diagram and cut it into pieces.

The anatomy of a lead gen landing page

Click-through landing page elements cut-out

To help you construct your click-through landing page, print out the following diagram and cut it into pieces.

The anatomy of a click through landing page

You are now ready to build your first landing page. (In whatever tool you are using as a landing page builder).

TAKE AWAY
When building your landing page, make sure every element of the page is working towards your goal. Remember to read your copy and media descriptions out loud to uncover areas for improvement.

3. Drive traffic to your page

This is a question that everyone asks – “How do I drive traffic to my landing page?”

The short answer is ‘marketing’. You need to promote your new page in the same way you would promote your company. If you don’t have a social media following or email list to contact, the fastest way to drive traffic to your landing page is by using cost-effective paid advertising like Google AdWords, Facebook ads or LinkedIn ads.

4. Gather insight into page performance

Once you’ve been driving enough traffic to your first page to see it’s conversion rate, you will want to start gathering feedback so you can begin the optimization process! Feedback can either come from users, or from colleagues.

User Feedback

Gathering user feedback is one of the most powerful steps in forming a hypothesis and testing new ideas. IT doesn’t get much better than potential customers telling you how to sell to them.

User feedback is pure gold when deciding what landing page elements to A/B test.

Inline Surveys

Get feedback from your customers at the most important point of the conversion funnel. Adding a tool like Qualaroo to your landing page allows your prospect to tell you how to improve your marketing messaging.

Live Chat

People have a natural degree of anxiety about finding businesses they trust on the web. There are a variety of methods that companies use on their website to minimize the anxiety, keeping their prospects in a state of mind where they can be converted. Live chat empowers you to answer questions prospects have, breaking down the psychological barriers to conversion. A live chat tool that many use is Olark.

Internal Company Feedback

You can also get feedback from others in your company (ideally those who don’t normally see your marketing landing pages).

5-second-test

The 5-Second Test

The 5-second test will test your headline to see if your value proposition is clear. A prospect should be able to understand your message within the 5 seconds it would take for them to make a decision to stay or leave.

How to run the test

Sit a person in front of a blank computer screen, then show your landing page to them for 5 seconds, after which time, take the page away. Then ask a simple question:

“What was that page about?”

If your headline is clear and concise enough, they’ll be able to explain the page’s purpose without having to read the extra information on the page. If not, revisit the test until your landing page passes.

If you want to run this test online, you can use a service called FiveSecondTest.com which lets you put your page in front of random subjects.

TAKE AWAY
Your target customers are the best source of insight for what to test.

5. Create a test hypothesis

A hypothesis is a statement of what you are going to test and your theory behind why it will be a success. It is designed to be proven or disproven when running your A/B test. It should be created based on the insight you have gathered and be targeted at improving the KPI you defined earlier in the process.

Testing and optimization research laboratory Marketing Experiments outlines a clever way to establish an effective testing hypothesis, based around three things:

  1. The presumed problem
  2. The proposed solution
  3. The anticipated result

You can use these to construct your hypothesis using the following formula:

Changing what your analysis indicates is the problem into what change you think will solve the problem will affect your key performance indicator is this way.

An example process for creating a test hypothesis

If you remember back to the KPI we defined for our fictitious flower store: “the number of bouquets sold online”, we can start to complete the hypothesis statement.

The presumed problem

Your online surveys have identified that the product images are too small to allow shoppers to get a real sense of how the bouquet will look in real life.

The proposed solution

Make the product images bigger with multiple angles.

The anticipated result

Shoppers will be able to see what they are getting more easily, leading to more online purchases.

Combining these within the hypothesis formula could lead to:

Changing the size and number of product images into something that allows shoppers to get a more accurate sense of how they will look like in real life when delivered will increase the number of flowers sold in the online store.

Now you can see a clear pathway to what you should be testing and your expected result.

TAKE AWAY
Taking the time to write a hypothesis allows you to reconnect the purpose of your page with your goals and KPIs. You can use a simple formula to create your hypothesis, which starts with identifying your problems through research.

6. A/B test your hypothesis

So you’ve got a landing page built, you’ve done your research to identify some problems, and you’ve written a hypothesis for how to optimize your page. The next step is for you to create a new page variant so you can implement your ideas.

An example A/B test

To illustrate a specific example, let’s return to the flower store and create a new test page. As you can see from the landing pages below:

  • The original (current champion) page has a small photo of the bouquet.
  • The new (challenger) page has an enlarged photo, along with instructions to click to enlarge, and a series of extra photos taken from different angles.

flower-store-a-th

flower-store-b-th

 
You now have a new page design created based on your test hypothesis, to test against the original page in your first A/B test experiment.

Other examples of landing page elements to test

There are a number of page elements you can change on your landing page for an A/B test, and many ways in which to change them. Below are a few of the more common approaches based on the page element in question:

  • Main headline
    • Promote different benefits based on information gleaned from user research and customer testimonials
  • Hero shot
    • Different photos – we recommend trying a photo of someone using your product in context
    • Video vs. photo
  • Call-to-action
    • Use the 8 guidelines of effective button design (from part 3 of the course: Call-to-Action design) to create a different variation
  • Forms
    • Fewer fields
    • Forms spread out over multiple pages
  • Page copy length
    • In-depth (long) vs. concise (short)

Some testing guidelines

In order to get definitive results from a test, you need to ensure it’s free of irrelevant data and that you’re using a large enough sample size. A test is considered clean if you’ve followed the guidelines below:

  • Each page variant should get at least 100 conversions without introducing any changes
  • The test has run for at least a week to account for different daily behaviors
  • Statistical significance should reached at least 95%, removing the potential that your results are based on chance

    Note: Unbounce and other testing tools will calculate statistical significance for you.

Now it’s time to have some fun. Once you kick off a test, you’ll often find yourself obsessively checking in on the stats. Sometimes you can get several people creating their own variants to battle against one another. It’s a great way to learn about your visitors and give people a sense of being involved in making the company being more successful. Just keep the guidelines in mind.

TAKE AWAY
Always use your hypothesis to guide your choice of page elements to test.

It’s important to repeat the process

There’s a natural feedback loop from gathering insight to running a test. Testing is a process that should never stop. Why? In simple terms, every page can perform better. You’re also not going to get it right every time. It’s important not to give up – always be testing.

Let’s test!

Think you know what your customers want? Think again. Gather feedback and form a test hypothesis before making any assumptions.

Remember to follow along with a free Unbounce landing page account

You probably already have a free Unbounce account by now to build your landing pages. If not, you can open a free Unbounce account (60 seconds to sign up – no credit card required). This will let you create landing pages for the examples we use throughout the course.

Watch this video to learn how to optimize your landing page though A/B testing

Not many marketers know how to do what you just learned!

Remember to jump into the comments and tell us what you think.


23 Responses

Add a comment

  1. Reply
    Karen
    Sep 19, 2013 - 12:39 PM

    Hi Oli,
    Thanks for the awesome lessons!
    I’m wondering about steps 3 and 4 under “Defining your goals and success metrics”. Is my KPI how I’m currently doing, and my target what I’d like to be doing?
    Thanks Again!
    Karen

    • Reply
      Oli Gardner
      Sep 19, 2013 - 06:00 PM

      Hi Karen,
      Your KPI is the metric you are measuring, and your target is the number you want to reach for that KPI.

  2. Reply
    Andy Kuiper
    Sep 25, 2013 - 01:30 PM

    makes sense – thanks Oli :-)

  3. Reply
    CNXTim
    Sep 26, 2013 - 12:38 PM

    Hi,
    I do need to convince a client to invest in an extensive Landing Page project, so looking forward to reviewing the “how to” of selling this to them,

  4. Reply
    hossein
    Oct 01, 2013 - 05:12 AM

    hi oli
    tanks for this lesson

  5. Reply
    Matt
    Oct 04, 2013 - 08:13 AM

    Hi Oli,

    Thanks for the course – finding these a very valuable resource to learn from.

    Do you always just test the one hypothesis (and therefore element) at a time? I understand the theory behind this (ability to empirically analyze the impact of each element change) – however if the landing page has many elements that are suboptimal then this could take many weeks for the full transformation to make. What is your view?

    Thanks!

    • Reply
      Oli Gardner
      Oct 04, 2013 - 10:42 AM

      Hi Matt,
      Excellent question.
      You are correct that you would ideally test them one by one.
      However, if this isn’t feasible, you could do a more radical test – where you do a wholesale change to the page.
      This should still be based on a hypothesis though.

      Here’s an example from a test that I ran:

      For the Unbounce homepage, I looked at our analytics and uncovered that only 1/15th of the traffic that visited the homepage even visited the features page.

      My hypothesis was that a large number of the people arriving at our pricing page were not as qualified as I’d like as they hadn’t consumed enough of our product information to really understand it. And by making a hybrid page with more of the features (written as benefits) we would receive a higher quality of customer.

      The results were very interesting. Neither page (the existing short page vs. the new longer and more detailed one) outperformed the other. This felt strange as it was a big difference in the level of information.

      Digging deeper into our funnel analytics however, we looked at more than just new trial starts, right down into product usage to a metric we call activation (customers who have completed 3 out of 5 tasks that mean they have become an active real customer).

      Looking at the activation stats, the long page had a 24% conversion lift in activated customers. Reinforcing that the new test page had indeed produced a more qualified customer.

      —-

      That’s the long winded way of saying that you can go bigger. Just have a good reason for doing so.

  6. Reply
    Tony
    Oct 12, 2013 - 08:21 PM

    Hi Oli,
    I’ve got a few questions regarding split testing for such a long time that nobody actually was able to answer them (or didn’t care to answer it). I hope you could shed some light on it cos I believe other people may have the same questions as well.

    My understanding is that “traffic” means “website visitors” which basically means people and you get traffic from organic or paid traffic sources.

    Here are my questions:
    1) what are your recommended time frames and timing for your landing page testing? Is it possible that you would get completely different or even opposite results if you do your testing on a different day, e.g. weekend vs weekdays? morning/evening? For instance, if you do your split test on a Wednesday evening and the result shows headline B is a lot better than headline A, but you do another test on a Saturday morning and the result shows headline B is a lot worse than headline A, what would you do?

    2) following my question above, how do you determine that if the test result is actually statistically significant? Would I have to run my test 24 hours a day for a whole week (because otherwise the test result could be random)? or for a whole month?

    3) following my second question, where would you draw a line? Nobody has unlimited testing budget and sometimes it’s just not feasible to run a split test for a whole week, not to mention a month. How do you set a proper budget for testing? e.g. if you sell a physical product and the profit is $10 for each successful sale, how much budget would you allocate for the split testing of the landing page?

    Your reply would be much appreciated,
    Tony

    • Reply
      Oli Gardner
      Oct 15, 2013 - 10:55 AM

      Hi Tony,

      1) You should be running your A/B tests for at least a week, preferably 2 – while paying attention to statistical significance at the same time.

      This is for exactly the reason you describe. There can be *huge* fluctuations based on the time of day or day itself. There can also just be a lucky swing in favour of one of your variants (where more qualified visitors happened to see one variant). This will be ironed out after time.

      I’m actually in the process of running a test right now, that showed a big improvement for one variant from Thursday to Saturday, and then completely flatlined on Monday/Tuesday. This is why you should wait.

      2) As mentioned, yes you ned to wait. Also, don’t be mislead by the statistical significance if it happens to get pretty high quickly, this too can fluctuate until your test has had time to settle in.

      3) This is the unfortunate part. You must let it run at least a week, otherwise you could be choosing a winner and then actually start losing more money if it would have turned out to be a losing variant.

      Just remember that at the end of the day, you will recover your ROI when you start running the winner on it’s own after the test, so it’s worth the wait.

  7. Reply
    Tony
    Oct 12, 2013 - 08:31 PM

    Hello Oli again!

    Sorry I forgot to ask this question in my last post:

    When running split testing, have you run into a situation like this and how did you (or would you if you haven’t) solve the issue?

    Let’s say you’ve run a series of split testing and here are the results based on the stats:
    (original landing page is comprised of headline A, hero shot C and CTA X. And you are testing a new headline B, hero shot D and CTA Y)
    headline B is better than headline A;
    hero shot C is better than hero shot D;
    CTA Y is better than CTA X

    Now you’ve determined that your new and optimized landing page would be comprised of headline B, hero shot C and CTA Y.

    What’s interesting is that now your new landing page performs worse than the original landing page—have you had any experience like that?

    Thanks again!

    • Reply
      Oli Gardner
      Oct 15, 2013 - 10:48 AM

      Hi Tony,

      That’s actually sounds like a Multivariate test, not an A/B test, as you are testing several things at the same time. Is that how you did it? Or did you run a series of A/B tests one after the other.

      One of the problems I find with multivariate is exactly what you are experiencing.

      The reason for this can be that together, the combination of the winning elements might not actually be as strong a unified message to your visitors.

      Let me know if you ran them at the same time or independently in sequence.

      • Reply
        Tony
        Oct 15, 2013 - 11:39 PM

        thanks Oli. I run them independently in sequence. Have you experienced similar results? cheers

        • Reply
          Oli Gardner
          Oct 17, 2013 - 01:56 PM

          I haven’t witnessed that myself.

          To recap:

          1. Headline B beats Headline A
          2. Hero C beats Hero D
          3. CTA Y beats CTA X

          What might be happening is that there wasn’t enough confidence in some of the tests, so you are building on the wrong result at some point.

          Or you could be driving a different type of visitor? Have you switched source at all? Say from PPC to email or banner?

          You could also have a message match problem, where if you’ve changed your ad communication (CTA, headline) it doesn’t correlate as well with the new page.

          Not a definitive answer I know. It is a strange situation.

          Also a sign that it is good sometimes to rerun a test.

          Do any of those scenarios apply to what you’re doing?

  8. Reply
    Kevin
    Oct 19, 2013 - 09:03 AM

    HI,
    We currently use HubSpot for our landing pages, but it does have limitations when changing elements if you don’t have css experience.

    Anyway we get around 40% conversion rate on pages, Is this good what would you consider to be a good conversion rate?

    • Reply
      Oli Gardner
      Oct 21, 2013 - 12:24 PM

      Hi Kevin,
      40% sounds great, but I can’t really qualify that answer as there are so many determining factors.

      For the most part it depends on:

      1. How targeted your traffic is. For instance, the conversion rate for registrants for this course was around 60% for our targeted leads. Compared to around 20% for general blog traffic that came through to the landing page.
      2. Is it lead gen or a click-through page? Any friction caused by a form will affect your conversion rate.

  9. Reply
    Eric Nedelman
    Nov 19, 2013 - 07:46 AM

    When you are doing A/B testing, is there any concern that people who refer other people may get the other page? For example: I tell my friend to check out a link, that the page has a certain offer, and give him/her the contact phone or email that I got on my page. If your other page has a different offer or contact information is there a concern that this may create a mixed message? Similarly, if I come back to your page a sometime later to commit to the offer I read during my first visit (when I did not convert) and the content is different, is there a concern that I will not convert on the new offer?

    • Reply
      Oli Gardner
      Nov 19, 2013 - 04:36 PM

      Hi Eric,
      Excellent questions.

      If you come back yourself, you will see the same page variant as you will have been cookied to ensure this. Of course, if you look on another device this wouldn’t be the case.

      For referring someone to the page, you are correct, they may see another version. The only way they would think it was a mixed message is if they saw the other one too. Although if you do have different offers that could be a factor.

      Typically you will have different messaging for the same offer, so the offer will be the same, just the way it’s “pitched” to the visitor is different.

  10. Reply
    Fernando
    Dec 24, 2013 - 05:27 PM

    Oli,

    Thanks for the course. My hole experience within Unbounce boundaries has been incredible. I believe I’ve read most of the ebooks (I just finished the one about infographics, which took me 2 hours because I read every reference you suggested) and it’s been great.

    I have a OptinMonster plugin which converts about 9-10% of the traffic into subscribers. With unbounce pages, I could increase that to 29, 35 or, more recently, 60% conversion associated with facebook ads. That’s pretty amazing.

    Thanks for all the great content. Keep up the good work.

    Cheers,
    Fernando

    • Reply
      Oli Gardner
      Jan 07, 2014 - 05:58 AM

      Those are great results Fernando! Thanks for sharing.

  11. Reply
    Deana
    Jan 04, 2014 - 12:12 PM

    Is there a base line amount of traffic a site needs to have before running an A/B test?

    • Reply
      Oli Gardner
      Jan 06, 2014 - 03:05 PM

      Not in terms of traffic no. You can start a test whenever you like.
      However, I wouldn’t just start a test for the sake of it (easy to fall into that trap).
      You should work on gathering some insight first so you’re not just making ideas up without any real foundation.

  12. Reply
    James
    Jan 14, 2014 - 08:20 AM

    Hi Oli,

    Really great course, thanks for all the insight.

    Perhaps slightly off topic on this page, but please could you shed some light on when you allow your landing pages to be indexed vs. not – for example they are very similar in content, so may be considered duplicate by Google if you are using the same frame work for you page and much of the copy (benefits, CTA etc.) is the same but the product or keywords (e.g different ad groups) is different? Thanks!

    • Reply
      Oli Gardner
      Jan 14, 2014 - 11:07 AM

      For the most part, I would recommend not indexing them – for exactly the reason you mention. But also, if you are doing paid search ads, you want to minimize the potential for people to arrive at your page organically as it will create a situation where your stats get poluted and your ROI measurements will be off. Make sense?

      If it’s an evergreen piece of content then that should be indexed.

Leave a Comment