UX Cambridge: Caroline Jarrett, How to find out about the usability of your website using a survey

Caroline Jarrett is a usability consultant. She tweets at @cjforms.

  • Recommended book: Letting Go of the Words by Ginny Redish
  • “Can a usability test just use a questionnaire, no observation?” (Audience: you need to see what they do, not just what they say; observation gives us context, avoids bias)
  • Surveys have inherent bias – people will say what they think you want to hear
  • It’s not a usability test unless it’s got some observation in it
  • The more interesting question is: Can we use surveys to assess usability of our websites?
  • Survey data can aid by triangulating between survey data and data from elsewhere (e.g. user testing)
  • Surveys can assess: effectiveness at task completion; satisfaction; demographics (stuff that can’t be tracked with analytics)
  • What is “the product” that we are assessing? Is it the website, a section, a page?
  • Net Promoter Score (NPS) — “Would you recommend us to a friend” — expands ‘the product’ up a level to be ‘the brand’, but┬ácan be horribly muddied if you try to fudge the question
  • Survey Tip: Ask one question at a time
  • Best thing you can do with surveys is looking at goals: “what did you come to this website to do today?”
  • Survey Tip: Find out about users’ goals
  • There are other goals to consider: What the organisation wants to achieve, What are our aims in doing a survey
  • Establish your goals for the survey (what are the questions you need answers to)
  • Types of survey goals: Exploratory (don’t know what is out there, so we send and hope or use a website popup. If you don’t know anything, any answers are better than no answers, as long as you don’t try using them as descriptive statistics with them); Comparative (explore trends, compare before and after – great for looking at trends, but you have to use the same questions every time, therefore you have to get the questions right first time); Descriptive (you know what is out there, go and count them — e.g. national census — got to do this properly with balanced sampling to avoid bias and measuring error); Modelling (find factors that show cause and effect – you’re seeing behaviours, trying to find out why)
  • You can combine types of survey – e.g. cohort studies use comparative, descriptive and modelling together
  • Examples of poor survey questions: poor defaults, double questions (two questions stuffed into one), options that we should already know the answer to via tracking, use of┬ájargon, grid of radio buttons (contributes to user dropout)
  • Bad surveys can have a negative effect on brand
  • Survey Tip: Interview first (talk to your users – good surveys always start with interviews, so you can understand how people want to respond to your questions)
  • Survey Tip: A successful survey is a process that involves loads of testing

One thought on “UX Cambridge: Caroline Jarrett, How to find out about the usability of your website using a survey

  1. Thanks Matthew. That was what I hoped I would say, so I’m pleased to find out that it worked. Cheers Caroline

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>