The 6 Reasons Medical Device Companies Skip Formative User Testing – and Why They Shouldn’t
Other than clinical efficacy, a medical device’s ability to meet its users’ needs is the most important predictor of success. For that reason, medical device development should begin with an assessment of user needs. It should return to those needs over and over again throughout the development process. Design inputs should map directly to user needs. And the design’s success in meeting those needs should be validated in testing. These formative user tests (also called formative research or evaluations) are the number one tool at your disposal to reduce the risk of designing the wrong product. They can spell the difference between a product that helps users meet their needs and one that breeds frustration. So why is it that so many medical device companies give formative user testing the short end of the stick?
The fact is medical device companies generally don’t skimp on user testing because they don’t care about whether their device meets the needs of users. They do it for reasons that seem perfectly rational. But anytime formative user testing is short-changed, it serves to undermine the development process — and the success of the resulting medical device.
Following are the six most common reasons medical device companies cut corners on formative user testing.
1. “We don’t need formative user testing – this medical device is just the next generation of our current product.”
Device companies often approach the next-generation update of a medical device with a very specific set of upgrades already in mind. Perhaps they’ve heard feedback from their sales team about something that customers aren’t happy with. Or perhaps they’ve developed a new technology they want to incorporate. Those are great places to start. But they’re not a substitute for having your human factors or design team interact directly with the users of your product.
After all, your sales team may have their own ideas about what needs to happen based on their interactions with customers. But unless you take the time to talk with your users directly, you’ll never know for sure what really should change. Giving users the chance to interact with your concepts for the next generation of your device becomes a foundation for a deeper conversation about your product. More often than not, that deeper conversation will surface new insights about how you should design your product.
2. “We know our customers. We’re the medical device industry leader.”
It’s great if your medical devices are dominating the marketplace. But the danger of sitting in this top position is that you may grow complacent in terms of seeking new ideas and innovations. You may forget the value of relying on your users to guide your development.
If you feel like you’ve already “heard it all,” then you’re probably no longer listening. And that means you’re no longer sensitive to the pain points your customers are experiencing. It doesn’t matter how well your products are performing. Your future success depends on how well you listen to your users in the context of formative user testing.
3. “We’ll conduct voice of customer right before the medical device launches.”
Voice of customer (VOC) is a marketing approach that typically uses questionnaires and interviews to identify a customer’s needs and expectations. Sounds a lot like formative user testing, right? VOC and user testing are similar, but they aren’t the same thing at all. And one is certainly not a substitute for the other.
This is true for two reasons. The first issue is that the definition of VOC is too nebulous. Different companies approach it in vastly different ways. The second problem is that VOC is typically geared toward understanding users from a marketing perspective rather than a development perspective. Formative user testing, on the other hand, is how you validate that every detail of a medical device conforms with users’ needs. VOC is concerned with the forest, while formative user testing is obsessed with the trees.
Too often, medical device companies skimp on formative user testing because they assume VOC will achieve the same goals. Compounding this problem is the fact that many companies wait to do VOC research until the homestretch of product development. Unfortunately, by that point it’s too late to make meaningful change. In many cases, what they are really doing is “validating” the design they’ve already finalized. Seeking positive opinions about the new product allows them to make the claim that “nine out of 10 doctors” prefer their device. In that case, VOC is really being used to support sales and marketing claims, not to more effectively shape a device’s design.
4. “We have clinicians on staff who will look at our medical device.”
Many device makers have physicians and other clinicians on staff to help guide the development of their products. The clinical input and involvement of these clinicians is crucial. But it’s no substitute for user testing. (That’s true even if your users are also physicians.)
Your staff clinicians are simply too close to your products to act as objective test users. Outside test users have the advantage of judging your design proposals for what they are. They don’t have any special incentives or past knowledge of the development process to cloud their vision.
In addition, your test users should be actively practicing in the field or setting that is most relevant to your device. Depending on how long your in-house clinical experts have been on staff (rather than practicing medicine), they may not be as current on the most up-to-date practices, treatments, and diagnostics. Who you want for your study and who you don’t will become crystal clear after you’ve created a detailed recruitment screener.
In order to design the best possible product, you need to get feedback from the people who are currently working under the most relevant conditions. You can count on them to give unbiased feedback, as they have nothing to gain except the hope of using a better product.
This isn’t to say that your in-house clinicians shouldn’t be involved in the development process. They should. Just don’t make the mistake of relying on their opinions alone.
5. “Formative user testing isn’t in the budget for this medical device.”
If you think a robust program of formative user testing isn’t in your product development budget, then you have a planning problem.
The reality is that user testing is always a smaller investment than the money you will lose if you take your product in the wrong direction. You could completely miss the mark with your product because you didn’t spend the relatively small amount on formative testing. At best, you’ll lose time and money as you backtrack to get your product headed in a better direction. At worst, you may end up producing a product that’s DOA.
Any new product should be planned with enough budget for sufficient user testing. One way to make sure your budget is realistic is to begin with a comprehensive project diagnostic. This important process guides your team in identifying all the essential project criteria, including formative user testing.
6. “We did do formative evaluations on this medical device!”
Finally, some companies aren’t even aware that they are short-changing formative testing. In this case, you may think you have mitigated your risks and checked formative user testing off the list. But ineffective user testing can yield misinformation that leads to a design change or pivot in the wrong direction. A new risk has emerged, and it’s not on anyone’s radar.
So how do you know you’re doing enough testing? There’s no magic number, but the more times you engage in a productive way with users throughout all stages of the development process, the more tailored your design will be to meet or exceed their needs.
In addition to frequency problems, many medical device companies that think they are doing formative user testing don’t do them right. The common culprits are poor planning, bad execution, or evaluating the wrong type of users.
Make sure you understand formative user testing best practices and stick with them. That’s the only way to ensure your user tests are pointing you in the right direction.
Everyone knows and agrees that it’s necessary to invest the proper resources in engineering when it comes to developing a new medical device. It’s time to cultivate the same mindset about user testing. And whatever you do, watch that your device company doesn’t fall into the trap of justifying a lack of testing.