Don’t Just Listen to Your Users — Watch Them, Too

 In Human Factors & Usability Testing, Medical Product Design

It goes without saying that medical devices should be both safe and effective. But those qualities are really just the baseline. If a product is safe and effective but doesn’t actually solve users’ problems, it won’t matter. To put it another way, the real measure of a medical device’s success is its ability to meet users’ needs. The most successful medical devices help solve users’ problems in a way that improves health outcomes and makes their lives feel noticeably easier. That’s true whether the primary users are doctors, nurses, patients, or consumers.

In order to create winning products, medical device companies must become the foremost experts on their users. To that end, user research is the foundation of good product design. Most medical device companies know this. Unfortunately, though, they often rely too heavily on users’ self-reported needs and behaviors. They take their users’ verbal responses at face value and use them to shape their products. The problem with this approach has to do with human nature: What users say is different from what they actually do.

Observational user research is the antidote. By looking at how users actually interact with a product or feature, you stand to gain a clear picture of the difference between what users say and what they actually do. And that allows you to surface key usability problems and opportunities that would otherwise go unnoticed.

Actual vs. Reported Behaviors: Why is There a Difference?

So why is it that users’ self-reporting can’t be trusted? Are they intentionally lying about their needs, wants, and behaviors related to your product?

The answer, of course, is no. Most users aren’t even aware of the discrepancies between what they say and what they do.

There are several reasons why this is so.

The first is that our memories aren’t as accurate as we’d all like to think. When individuals self-report, they may give incomplete or inaccurate information simply because they don’t remember all the details. For example, if you ask a doctor to describe the steps she uses to perform a procedure, she might forego details in retelling that she wouldn’t forget if she were actually doing the work.

The second reason for inaccurate self-reporting has to do with the fact that people have a natural desire to be right. In the context of an interview, they want to be perceived as knowing their stuff. And that may be especially true when you ask them about their jobs. Because of this tendency, users are less likely to admit it if they aren’t sure about a specific procedure or process. They may err on the side of too much certainty subconsciously. Either way, the result is less-than-accurate information.

Finally, healthcare workers are trained to perform tasks in a very particular way. When you ask them how they execute that task, they may naturally default to the standard operating procedure. They may report one set of behaviors when, in reality, they also use workarounds or take shortcuts that make their jobs feel easier.

For example, healthcare workers are often trained to use personal protection equipment (like gloves) when performing certain tasks. In self-reporting, these users would tell you that they use the gloves. But in reality, they may have found that the gloves limit their dexterity. Because the gloves make their job more challenging, they may not actually use them the entire time. That’s something you would never know without observing your users in action.

In this case, the knowledge that practitioners don’t always comply with glove usage presents an opportunity. It tells you that you should strive to make your product safe without the use of gloves or easier to use with gloves on.

Incorporating Observational User Research into the Development Process

Observational user research isn’t a one-and-done proposition. The FDA provides guidance on user testing requirements. In certain instances, only summative testing is required. However, to get the best results, we recommend incorporating observational user research at several points along the medical device development process. Each round of observational user testing has a distinct purpose. Taken together, they ensure that your medical device is truly tailored to meet your intended users’ needs.

Use the following scheduling template to guide your planning:

  • Phase 1: Field research. Field research is conducted during the discovery phase. In this round of observational user research, your goal is to gain a general understanding of your intended users’ environment and work habits. Observing your users in a more open-ended way (without directing them in any way or asking any questions) gives you lots of important context that you can apply to your own product. For example, you might discover that your users regularly do certain workarounds because of their particular environment (such as taking a shortcut that prevents them from having to walk across a large facility to retrieve something).  Those workarounds become opportunities for product improvements and innovation.

  • Phase 2: Contextual research. The next step is contextual research, which also takes place during the discovery phase of medical device development. During this phase, your goal is to get more specific about what matters most to your users by engaging with them more directly. To begin, observe them performing or demonstrating particular activities that relate to the product you intend to design. Next, ask questions about why they approached their task the way they did. Doing so gives you early insights into what users are saying they want versus what they really need. For example, a user might tell you they like a particular version of a device. In practice, however, you may notice that they always reach for the older version in the drawer. The explanation may be as simple as the older device is more ergonomic. Preferences like this one give you an important understanding of the features and qualities your own device should possess.

  • Phase 3: Formative research. Once you’ve developed an initial concept or prototype, it’s time for formative research. During this phase, you observe your users interacting with your concept or prototype in a simulated or actual environment. In some cases, you might even present multiple iterations for the users to provide insights and feedback on. Formative research gives you important early insights about how your users will actually engage with your product. The things you observe them doing — such as struggling to perform a function or more quickly solving a problem — will help you refine your product’s design.

  • Phase 4: Summative research. The final stage of user testing takes place when your product is nearly finished. Summative testing is more rigorous and technical in nature. The goal at this stage is to validate your product’s design. You must have production-equivalent systems in place. In addition, you must first train your users on your product just as you would with your final end-users.

Your product’s success depends upon how well it meets your users’ needs. But don’t take their word for it. If you want to get it right, watch them in action.


Recent Posts