User Testing, Usability Testing – What’s the Difference?

We’re often asked for our thoughts on user testing and usability testing and sometimes find it hard to explain that there’s a real difference between the two (even though as phrases they sound similar).

• User Testing – is there a need in the marketplace for your product/service/app?
• Usability Testing – how well do users get on with your website/mobile site/app?

Both types of testing add real value to the design and planning stages of any new or existing product/website/app, but at different points.

User Testing

This ideally needs to be carried out thoroughly before any real design or implementation work is undertaken, as the findings from this research can drastically change the original proposition. It may be that a good idea on the face of it actually turns out to already exist, or the marketplace is so flooded that it would be too difficult/expensive to make a real impact.

This doesn’t mean that conducting user testing will always show up the negatives; indeed, some really good services have come from even the simplest of user tests. It gives the opportunity for both the proposer and the development/design team to really understand the needs of the potential user and tailor the product/service accordingly.

Examples of User Testing

• Uber founder Travis Kalanick originally came up with the idea of an app for “cabs on call” when he was stuck in Paris in the rain and couldn’t get a taxi for love nor money. To “user test” his idea, he visited different bus stops/taxi ranks/building exits and asked a wide range of people if they would utilise this kind of app. The answer was obviously yes, as today Uber is one of the most popular services in the world (London notwithstanding!)
• Airbnb founders Joe Gebbia and Brian Chesky were both struggling to pay the rent on their shared apartment, so came up with the idea of renting out 3 airbeds at $80 a night in their living room for guests of an upcoming design conference in San Francisco, with cooked breakfast included (we should be so lucky these days!). When this was successful, it was a natural decision to push the idea further. Both men travelled to various conferences and festivals across the US, broaching the idea with locals about listing their rooms and also using the service themselves. By the time the Democratic National Convention 2008 rolled round, they had the infrastructure in place to launch the website and within a week had 800 listings!
• Dating app Tinder divides opinion but there’s no denying its massive success in the social media and dating arena. Designed by two college graduates as “the future of dating in the digital world”, they spotted an opportunity for a smartphone only avenue that would appeal to users aged 18-24 (although this has now expanded and this demographic only makes up 50% of the overall user base). To test the idea, they pitched it to other college kids (initially a group of 300 at USC) as to whether they would use it and whether the idea of swiping right to find a match would appeal more than filling out lengthy bios without connection to other social media profiles. When this was a hit (over 1000 sign ups in a week), they then seeded it across other college campuses to get the initial user base on board. Today, it boasts such stats of 1 billion matches made and 800 million swipes per day, so they obviously did something right.

Usability Testing

Usability testing is the stage that should come AFTER user testing, and ideally once a rough platform/app is in place to actually test on real world potential users. This means that the two don’t necessarily follow each other; the flow should have design and development in between, although these will be no means concrete until after usability testing is performed (and ideally more than once!)

Usability testing will help you to understand how effective your service/app/website is, where people may get “stuck”, anything that doesn’t work (aesthetically or functionally) and also allow you to form recommendations on fixing the issues before any further usability testing is conducted.

Aesthetic changes will often be a matter of preference but there are some core rules that many businesses tend to stick by, such as using “cold” colours like blue and white for corporate sites, and using enquiry forms to make things easier for users. Other elements, such as logos, font selection and imagery, may appeal to some and not to others.

You don’t need to initially have a fully fledged website/app or product in order to conduct usability testing – initial stages can be done with basic tools such as rough mock-ups in PowerPoint to gauge initial reaction. Further down the line though, we’d recommend putting a/b split tests in place, especially for websites where even small changes could have big effects on sales and performance.

Usability Testing Examples

• Going back to our Uber example above, Travis Kalanick initially created a rough app and asked his friends to try and use it. They couldn’t, there were functionality issues that prevented them booking their cab. With this in mind, he fixed the flaws, asked them to test again, and then further fixed any resulting issues from this. It’s all about doing it, and then doing it again (and again, and again…) until the majority of users have no issues (as with all walks in life, there will always be one who doesn’t like something, and you can’t please them all!).
• Apple were conducting usability testing as far back as 1982; the manual for Apple software developers for that year advised that they should select their target audience, determine how much they know about Apple computers and the subject matter, and then design the user interface to suit the audience needs. Usability testing was advised to be on a wide range of user groups, including friends, relatives and new employees, without telling them that it was software, rather than hardware, being analysed. Apple’s designers had to watch their user groups in person, to see if they were expressing frustrated body language (such as ‘deep heart-felt sighs’ and ‘shaking heads’).
• Social media giant Facebook are well known for their usability testing methods; all too often we see the interface change overnight to move layouts and colours, with an option to revert and give feedback on what we like and what we don’t. As frustrating as it can be to find your News Feed has disappeared (along with your “other” messages), eventually Facebook will either concrete these changes or remove them; this is all part of their ongoing quest to deliver the perfect personalised social media experience.