Get my ecommerce book (and 20 others) for just $0.99 on Wed 12 and Thu 13 May only!

☰ View all evidence-based methods

Remote User Testing

What you can learn

User testing is arguably the most useful evidence gathering method of all. In my experience, the number of ideas for site improvements that come out of a session of user testing surpasses any other method.

If you've followed the process of gathering quantitive data first and you know you have conversion issues on your site, this can tell you why issues have been happening. You'll be able to watch users go through your flow and (providing your test is well designed) you'll see where they get stuck, and hear them tell you why they don't like or can't find something.

You can gain this knowledge from approaches such as lab user testing or even guerilla user testing. However I find remote testing has a few big advantages:

There’s no excuse not to set up a quick unmoderated remote test with a few users for each major feature you redesign or as a regular monthly/quarterly check-in.

How to do it

There are three main methods of remote user testing: 1. facilitated & moderated by you; 2. facilitated (and possibly moderated) by someone else; 3. facilitated by you but unmoderated. Each of them works a bit differently, which I’ll explain here.

Facilitated & moderated by you

For the moderated & facilitated by you approach, this obviously involves the most work for you but can potentially cost nothing. You'll need to find the users, organise a time to have a video call, record the call, and then write up notes. I will ask clients to suggest users for me to contact, and will email them to book in a time for a call using the handy Calendly.

The call itself consists of using Skype or Zoom so they can share their screen with me. I then share a link to a prototype or website and can see and hear them as they navigate it. The call can be recorded with screen recording software (on Mac QuickTime is handy for this) and immediately after I write up my main observations.

Facilitated by a third party

The facilitated by someone else approach means hiring a company to set up and run your user test, which may or may not be moderated by them as well. Moderation is generally useful when you're testing a prototype or early version of something that requires a bit of explaining or isn't fully working.

Either way you’re role is to specify what you want to test, and liaise with them as they develop the test. They'll then run and analyse it so you get a report at the end with the findings.

Facilitated by you and unmoderated

The unmoderated option consists of you setting up the test and putting it out to a panel of users who are ready to go. You then get back the videos of the users navigating the site for you to analyse and draw insights from. If you're testing a live website I think unmoderated is the best way to go as this is closer to the reality of how users actually browse the web.

You will need to develop some skills in putting together a decent test and you’ll need the patience to watch videos of people going through your site. As painful as this can be at times, as a UX designer or product manager, there are few better ways to understand what your users face.

Watch out for

An important part of writing a user test is to make sure you're not putting leading instructions in there. Like leading questions when surveying, you don't want to be pushing the users to do certain things or you'll never learn what they would naturally do. Keep tasks simple by saying things like 'show how you would search' rather than 'click the search button in the top right and fill out your dates and location'.

Some people simplify the whole test by only setting users one task like 'show how you would buy a product'. The danger here is that users whizz through the process and you don't get to see them interact with all parts of your site, hence why I prefer a bit of guidance with a task per step of the flow I’m testing.

Make sure you recruit accurately for your tests. You'll want people who match your actual users (you can use your audience data to discover this). It’s very rare that a website is designed to appeal to absolutely everybody so you want users who are going to provide authentic feedback.

Unless the flow is very complex, I've found testing with five users gives plenty of feedback and improvement ideas—adding more just tends to see repeated behaviours. Make sure you have five per major device category though, as people can behave very differently on each them. For example, I most commonly test with five on desktop and five on mobile.

When it comes to analysing your own tests try and stick to recoding observed behaviours. Users might *say* that they don't like a feature (especially if it is new) only to be perfectly competent at actually *using* it. Quotes are useful to put in reports to explain behaviours but shouldn't be used if they don't reflect what actually happened.

Aim to watch all your tests through and annotate them first so you have a good sense of events, before summarising the repeated insights and critical issues in a lightweight report.

Tools (and cost)

For your own moderated desktop tests, a decent free option is just good old Skype along with recording on Quicktime.

When it comes to unmoderated testing platforms who recruit for you, there are several pay-as-you-go options with different features to suit all budgets. Here are a few I've used:

If you want someone else to recruit and facilitate and you have a large budget, look at bespoke but pricey services such as UserTesting, WhatUsersDo, and UserZoom.

How long does it take?

Unmoderated testing can be done in a matter of 2-3 days for 5-10 user tests. If you're moderating it yourself then the extra organising tends to mean it takes about a week.

How often should you use it?





Last updated on 13 May 2020

Save time with this

User Testing Report Template

User Testing Report Template


(+VAT in EU)

For Keynote, Powerpoint & Google Slides

A template for creating lightweight user testing reports. 11 pages including test outline; test participant details; executive summary; bugs list; and sheets for key findings (including usability problems and positives).

Multi-pack: buy 6 templates & save 65%!

$22 $7

.50 (+VAT in UK/EU)

All of the planning and reporting templates – includes the Keynote, Powerpoint and Google Slides versions – with all the walkthrough videos and extra content.

☰ View all evidence-based methods