Collect Your Data

Learning Objectives

After completing this unit, you’ll be able to:

• Collect data in a way that avoids bias.
• Observe critical user behaviors.
• Interview your users about their actions and needs.

Build Your Interview and Observation Skills

In previous units you learned how to lay the groundwork and prepare for studying your users. In this unit, we’ll cover the fundamentals of observation and interviewing.

When interviewing, what you say and do has as much of an impact as what you choose not to say and do. As a result, it may be stressful in the beginning to stay on script, ensure you cover everything you need to, and take good notes all at the same time. But with practice, this will become second nature. Carla uses a checklist to help her remember all her UX research prep work.

Checklist
Things to observe Question list/script Audio or video recorder Pen and paper Small thank you gift

A Researcher and a User Walked into a Room…

After introducing yourself and the purpose of your research, use the first part of your session as a way to build rapport with your participant. Remind them that they’re free to end the session at any time. No one should be coerced into participating.

Carla knows the more comfortable her participant Dana is, the more she’ll share with her. She assures Dana she isn’t there to evaluate her, but rather learn from her and make her life easier. She focuses the first few questions on learning about Dana and her role. For example:

• How long has she been in this role?
• Can you describe for me what a typical workday looks like?
• What are the biggest challenges she faces with respect to her daily work?
• If there is anything she could change about her job, what would it be?

Once you develop a rapport with your participant, you can move into the core of your research activities: observation and interviewing.

Tips for Conducting UX Research

•  Be approachable and positive—Remember to smile! Your participant may be nervous and concerned that they're being evaluated so provide encouragement along the way while remaining neutral (for example, “This is really helpful to hear!”).
•  Ask questions—Don’t assume anything. Ask questions about anything that isn’t clear.
•  Stay focused—It’s easy to find things in common when you’re building rapport with your participant, but you have limited time, so you want to make the most of it. If your participant begins to stray off topic, gently bring them back on track. Keep an eye on the time and what you have left to cover.
•  Remember, you’re not a participant—This activity is about your users, not you. It’s their experience you’re studying, so it’s not a good idea to share your own opinions or experiences. It’s natural for your participant to ask your opinion; just let them know you want to hear from them first, and save their questions until the end of the session.
•  Resist the urge to train—Even if you disagree with an idea or request a participant makes, keep it to yourself. If you observe that the participant is doing something “wrong,” now is not the time to correct them. You can share your recommendations with them at the end of the study, but only if it’s absolutely necessary. You don’t want them thinking you were actually evaluating them the whole time.
•  Watch your participant’s body language—Pay close attention to your participant’s body, demeanor, and tone of voice. If a particular line of question is uncomfortable, if the task you’re observing is particularly stressful, or if your participant seems tired of the study, change your line of questioning or end the session. Even if the topic/task you’re studying is super important, if you force the situation, you won’t get useful feedback.
•  Be flexible—Things don’t always go as planned. (#word!) Your participant may be late, your video camera battery may die, or the situation you’re there to observe (for example, how Dana handles an incoming sales call) may just not happen during the session. Have backup plans for the most likely scenarios. For example, take extra batteries and have a list of backup questions or activities if the events you want to observe aren’t happening while you’re there.

Beware the Thorny Hawthorne Effect

We all know the feeling of having to perform a task with someone watching over our shoulder. It’s not a good feeling, and it often affects how well we perform the task.

Now imagine that how you perform the task affects the future direction of a product, process, or service. You can imagine that your behavior might not be the same as if you were not being observed. This is known as the Hawthorne effect (Landsberger, 1958), and its effects are greater when there’s a difference in power between you and the person you’re observing. The good news is that the effect can diminish over time, especially as you develop a rapport with the participant—another reason to build rapport at the start of your session.

Become an Outstanding Observer

In many cases, a simple interview doesn’t provide enough information to understand how your users work, what they need, their pain points, and workarounds. Putting yourself in the user’s shoes gives you the context needed to better understand who you're designing for. The most popular method for this is called “contextual inquiry.” Beyer and Holtzblatt’s 1998 book, on contextual inquiry (CI) and contextual design, outlines four main parts to contextual inquiry:

•  Context: Go to the users’ environments to understand the context of their actions. In this case, you probably want to visit your users at their desks, rather than in your office or a conference room.
•  Partnership: Develop a master-apprentice relationship with your participant to better understand them, their tasks, and the environment they work in. Immerse yourself in your users’ work by performing the tasks they do, the way they do it. For example, taking customer calls or resolving support tickets.
•  Interpretation: Interpret your observations with the participant. Verify that your assumptions and conclusions are correct.
•  Focus: Develop an observation guide (that is, a list of behaviors, tasks, and/or areas to observe and questions to ask) to keep you focused on the subject of interest or inquiry.

Put Your Best Interviewer Foot Forward

Sometimes, you can’t observe your users, or perhaps like Carla, you’ve already observed them. This is where you can do stand-alone interviews. Here are some things to keep in mind.

•  Find a quiet space: If you won’t be conducting observations, then avoid distractions and interruptions by using a quiet office or conference room for your interviews. Try to stay close to the participant’s desk in case they want or need to reference something to answer your questions.
•  Stick to the script: Hopefully you’ve already developed your interview questions and checked them for all forms of bias (for example, leading questions). In the moment of the interview, it’s easy to go off script and word your questions in a biased manner. This can happen to even a seasoned researcher, so avoid this and stick to your script.
•  Focus on outcomes, not solutions: When you interview your users, they’ll most likely provide solutions to the challenges they’re facing. They may not be optimal, or even technically feasible, but it isn’t your participant’s job to come up with the solutions—it’s yours! So instead, focus on the outcome. What’s the goal they want to achieve?
•  Develop patience with silence: Silence can be uncomfortable, and most people are eager to fill the silence or “help out” by suggesting words or completing their user’s sentences. By doing that, you become the participant and introduce your own bias into your research. Use the silence as a signal to your participant that you really do want to hear what they have to say. If you’re taking notes, stop and make eye contact, so your user sees that you’re paying attention. Use acknowledgment tokens, like “uh huh” and “OK” to acknowledge that you’re listening.
•  Watch for generalities: People often think it’s more helpful to talk about how things usually happen or they talk about “most people.” These generalities are less helpful than hearing about specific instances your user has actually encountered. You have no real way to know how often “usually” actually is, if your user’s assessment of what “usually” happens is an accurate summary, or if they really know what other people do. So when you hear them, probe for more details or actual examples.
•  Allow room for “no opinion”: Sometimes your user really has no opinion about a topic, or doesn’t know the answer to your question. That’s OK. No need to force them to make a guess. Just move on to the next question.

Notes on Note Taking

You’ll likely want to take some kind of notes during your activity, whether it’s with pen and paper or on a laptop. A potential problem with taking notes is that you can get so wrapped up in being a stenographer that you stop engaging or following up with your participant. The last thing you want is to miss a critical comment because you can’t type fast enough to keep up.

Carla isn’t a fast typer, but she has developed a shorthand so she can note key elements quickly. She also doesn’t try to capture every comment verbatim. She knows paraphrasing is faster and sufficient for most purposes. When it’s important for her to collect verbatim quotes, she uses video or audio recording. Then she simply notes the time an important observation or comment was made, and she can quickly go back to these points in the recording.

Thanks… You’ve Been a Great Audience

Like most conversations, you want to leave on a good note. So as Carla completes her observation and interview with Dana, she thanks her for her participation. She’s heartfelt in her gratitude for the time she’s spent with her and for her invaluable observations. Before she goes, she asks if she has any questions for her.

She also gives her a quick summary of her next steps. She has a small gift ready (a coffee mug with the company logo) to show her appreciation.

Great, our data gathering is over and we’re ready for the next unit, where we’ll look at how to collate the data you’ve recorded and look through it to identify any trends, commonalities, and differences across the users you’ve interviewed.

Resources

• Understanding Your Users, Second Edition: A Practical Guide to User Research Methods (Interactive Technologies)
• Holtzblatt, K., & Beyer, H. (2009). Contextual design. Human-Computer Interaction: Development Process, 55.