Tag Archives: experiments

Can a design be too good?


Panda background on bing
Panda background on bing

Bing recently designed this lovely homepage that had a video of a panda chewing some bamboo in the background.

Because I work in digital and for WWF this is exactly the type of thing I tend to get a few emails about.

It was lovely… I spent a good while watching it… and then left their site without making a search.

This got me thinking: does amazing design inspire action, or hinder it?

That is to say: does that flashy video or the inspiring image you’ve added to your site distract the user – your visitor – from the asks and actions on the rest of the page.

This isn’t a new idea – this is something that’s been tested a million times over.

We know different images, sizes, positions can create different results… however – what I’m considering here are the ‘flashy’ (not Flash) things.

At what point does design become too good?

If I had the chance to I’d love to see if the search rates on Google are effected when the Google Doodle is interactive.

Experimentation time

I can see how Bings users reacted to their little bit of integrated video. Neither can I see how Googles search rates are effected. What I can see is how our users might react.

So I put together a video background banner for our homepage for a tiger-takeover week we were running to celebrate World Tiger Day.

Video tiger banner
Video tiger banner on the WWF website homepage

Its pretty simple and uses the YouTube API (tutorial coming soon). If you’re wondering it has a fall ball for mobiles and browsers with JS turned off.

But how and what do we test?

Let’s keep it simple shall we – let’s test the banner with the video v one with just an image. And – to keep it even simpler I’m going to use a third party service (Visual Website Optimiser) to switch the video in and out.

I could use Visual Website Optimiser‘s tools to track it – but really I want some of the additional metrics that Google offers – so I’m going to fire a bit of event tracking code when the video is loaded. This will allow me to segment out the two streams of traffic on GA.

And what to measure?

I have an engagement score for the site. Its a pretty basic metric but it works as a simple measure of success when compared with itself on other pages, sites and time periods. In simple terms it binds and normalises some of the key site measures (such as time on site/page,  bounce rate, pages per visit). Basically – if page one has an engagement score of 3 and page two has a score of 6 then page two is more engaging.

Using the Google events we’ve set to fire earlier we’ll be able to see which variation has the greater engagement score.

So that’s that set up… and it’s currently running! Watch this space and I’ll post some of the findings later.