Tag Archives: a/b testing

Can a design be too good?


Panda background on bing
Panda background on bing

Bing recently designed this lovely homepage that had a video of a panda chewing some bamboo in the background.

Because I work in digital and for WWF this is exactly the type of thing I tend to get a few emails about.

It was lovely… I spent a good while watching it… and then left their site without making a search.

This got me thinking: does amazing design inspire action, or hinder it?

That is to say: does that flashy video or the inspiring image you’ve added to your site distract the user – your visitor – from the asks and actions on the rest of the page.

This isn’t a new idea – this is something that’s been tested a million times over.

We know different images, sizes, positions can create different results… however – what I’m considering here are the ‘flashy’ (not Flash) things.

At what point does design become too good?

If I had the chance to I’d love to see if the search rates on Google are effected when the Google Doodle is interactive.

Experimentation time

I can see how Bings users reacted to their little bit of integrated video. Neither can I see how Googles search rates are effected. What I can see is how our users might react.

So I put together a video background banner for our homepage for a tiger-takeover week we were running to celebrate World Tiger Day.

Video tiger banner
Video tiger banner on the WWF website homepage

Its pretty simple and uses the YouTube API (tutorial coming soon). If you’re wondering it has a fall ball for mobiles and browsers with JS turned off.

But how and what do we test?

Let’s keep it simple shall we – let’s test the banner with the video v one with just an image. And – to keep it even simpler I’m going to use a third party service (Visual Website Optimiser) to switch the video in and out.

I could use Visual Website Optimiser‘s tools to track it – but really I want some of the additional metrics that Google offers – so I’m going to fire a bit of event tracking code when the video is loaded. This will allow me to segment out the two streams of traffic on GA.

And what to measure?

I have an engagement score for the site. Its a pretty basic metric but it works as a simple measure of success when compared with itself on other pages, sites and time periods. In simple terms it binds and normalises some of the key site measures (such as time on site/page,  bounce rate, pages per visit). Basically – if page one has an engagement score of 3 and page two has a score of 6 then page two is more engaging.

Using the Google events we’ve set to fire earlier we’ll be able to see which variation has the greater engagement score.

So that’s that set up… and it’s currently running! Watch this space and I’ll post some of the findings later.

What should a homepage look like?

Homepages are funny things…

The BBC homepage
Oh, I do like the BBC homepage

Over the years I have seen some odd behaviour when it comes to the company homepage. In a number of organisations now, some people (colleagues) have held the view that the home page as a promo opportunity (which it is, I suppose) – sometimes vying quite fervently for space as the life of their activity or campaign depended on it (it doesn’t…often).

And I get why:

  • It often acts as a shop front for the organisation
  • Items featured on it seem – by implication – important
  • It does (often) win the crown for most visited page on the site

However – what people often forget – and I actively remind people – is:

1. Not everyone will see your home page.

For us its about 20% of all visitors. Most visitors (4 out of every 5) will have followed a link in an email, or made a search on Google or clicked on a social post and have ended up directly at the piece of content (and page) they require. Which brings me onto my next point…

2. Visitors have a reason that they’ve arrived at your site.

The (1 in 5) visitors who arrive at your homepage haven’t – for the most part – just stumbled upon it to see what’s going on. I realise that’s a broad, sweeping statement and there are definite and obvious exceptions (BBC news, Facebook, etc) – but for most of us this is true. Visitors come with a purpose and – even if they’re one of the few that land on the homepage – they’re going to be making their way to their desired destination.

So what is a homepage for?

The pebble homepage
The pebble website has a single purpose and the homepage reflects this

Well for some it will be obvious… if you are – for instance – Pebble then you’re going to use it as your shop front for Pebbles.

But if you’re an organisation like – say WWF – then there are likely one of over a hundred reasons that someone might have come to the site – so you have two options:

  1. Make your homepage the first level of your navigation structure
  2. Use your homepage as promotional space for activities / campaigns / products that visitors might not have known about but might be interested in (but obviously don’t forget the nav)

I’m going to focus on the second option for now as… well – I think its where most time is worth spending.

In my day job I’ve spent a lot of time reticently multivariate  testing elements of our homepage to get the best results and I thought I’d share some of the results.

NB: Each site’s audiences are different and it’s important not to take my word as gospel – but please feel free to take my findings and test them yourself. I tend to use either Google Experiments or Visual Website Optimizer for my testing.

A single – well formed hero space – works better than a carousel.

I spent a lot of time testing different versions of the hero space on homepages. It seems that having one reduces bounce rate (my theory is that if the imagery is right it acts as confirmation for visitors that they’ve landed at the right place) – but carousels traditionally get very little click through.

The tool being used to countdown to Earth Hour on the WWF website
The tool being used to countdown to Earth Hour on the WWF website

Carousels muddy the water. They don’t give a persuasive argument for one action over another. Think of it as a salesman… (good) sales people don’t go out with 50 items and list them off to potential punters – they have one product and they use all their effort to sell it.

So with that in mind put in the effort to “sell” that ask. Movement works well (unsurprisingly) as it draws the visitors attention. Recently I applied a countdown and that did EXTREMELY well. It turns out (I assume) people panic when exposed to a number slowly creeping to zero.

Big images and less text.

I’ve worked at a number of organisations that have used the first paragraph of a news article or blog post to “persuade the visitor to read more.” This is a nice thought but it relies on the user reading that first paragraph in the first place… and (not to over simplify things) but if we can avoid it people try not to read.

After much testing it turns out a large image with a small bit of copy works best. Make sure the copy changes colour on hover… oh and make sure the whole area is the anchor… buttons work well – but not as well as if the user can click anywhere within the space.

In some instances I’ve even found just a heading and image performs well – especially if the heading is a clear call to action: “Take this quiz” or “Sign this petition” for instance.

Men click icons, women click copy.

A couple at a computer.
A couple at a computer. Poss. looking at my website?! Image by Ed Yourdon

This is more of a general thing but something I found whilst doing demographically segmented heatmapping is that men tended to click icons and images. Women on the other-hand tended to click on text links more.

How is this useful? Well – if the audience you’re trying to hit is of a specific gender then its important to use a link type they’re statistically likely to click. Also it can be important to remember that one way isn’t always best. Armed with this knowledge I tend to ensure important links include both copy and iconography.

Don’t forget the navigation.

Remember that your visitor has come to your site for a reason – and your aim shouldn’t be to corner them into a position where their only choice is to click one of the few options you’ve put in front of them. They won’t. They’ll leave (and increase your bounce rate).

A good nav is important… but don’t forget to monitor your analytics and check to see if you might be getting any spikes as a result of outside influences (you want to be looking at the organic search keywords). I’ve found conv. rates increase for asks when external activity is happening.

Add a link direct to that ask on the homepage and you’re removing clicks, barriers and drop-off points between the user landing and finding something relevant.

The last thing I want to mention is not to forget the 80% (“we are the 80%” I hear them cry) of those who don’t visit the homepage. Finding other locations to cross link within your site – especially where it’s relevant – is crucial. I’ve tested some wording and found “People who read this also visited:” works well as a way of promoting your cross-links.