FDA Finalises Human Factors Guidance – summative testing – Part 3

In this series of posts, we’re taking a look at the recently published final version of CDRH’s guidance “Applying Human Factors and Usability Engineering to Medical Devices”.
We have already taken a look at the first three in this list of the key topics covered by the Center:

  • Human Factors Engineering processFDA HF guidance FINAL
  • Risk Management and Human Factors
  • Design Verification
  • Summative Human Factors testing
  • Changes to products already on the market
  • Human Factors Engineering Summary report

In part 3, we’ll look at how the topic of Human Factors testing (both formative and summative) may impact what you’re either already doing, or plan to do.

Formative Human Factors testing

Look upon this stage of testing as providing a toolkit to identify potential use-related hazards and hazardous situations. The agency emphasises the value of testing with participants who are representative of your intended users, as this yields early data on how your eventual users cope (or don’t) when using your prototype device.

The agency recognise that formative evaluations provide information about;

  • Design of the user interface (those parts of the device that are manipulated by the user, provide them with information or indicate function)
  • The effectiveness of features or changes implemented to reduce or eliminate use errors (otherwise known as mitigations)
  • The the need for training of users and inform the design of training materials
  • The design and content of labelling such as the Instructions For Use, product labelling, packaging.
  • The content and design of your summative testing.

In the guidance, the Center talk about the value of effective formative testing in preventing very costly errors in the design of your validation test protocol that render it just another formative study.

In a departure from previous guidance, there is information on the expected content of formative test protocols, which creates a coherent and building picture that results in the information now expected for your HF validation test protocol. It’s very much a case of using formative studies as building blocks that provide a robust foundation for your validation test.

Summative Human Factors testing

Summative testing is still in place. However, testing that demonstrates the “user requirements” or User Needs have been met will henceforth be referred to as “Human Factors Validation Testing”. One could perhaps see this as coming into line with the philosophy applied to validation of processes, test methods, production lines, etc

No longer does guidance expect you to just include critical tasks within your test design. Rather, the testing needs to be comprehensive, so that the results can be related to actual use.

Test participants

If your device is intended to treat patients with a condition that can cause them to have functional limitations, these conditions must be represented during both the formative evaluations and your HFE validation study. If you choose not to design your device to accommodate needs of those likely to use your device, then labelling should clearly explain the capabilities that users must have to use the device safely and effectively. You should, therefore, ensure that you’ve understood your Users’ needs thoroughly and scoped out the Usability Specification appropriately to know what capabilities/limitations are to be included in study participant groups.

Participant Training

If you anticipate that most users of your device would receive minimal or no training, the Center now expect that participants in your validation study do not receive training.

A period of time between training and participation in your validation study should be built in to the study design. How long that is will vary, depending upon what would happen in real life and the impact of training decay on the use-related risks that you’ve identified for your device. The important thing is to consider this topic and document your rationale for the chosen approach.

Study protocol

Manufacturers are now formally encouraged to submit draft HF Validation Test protocols for agency review, to ensure that the methods you plan to use are acceptable (via the pre-submission programme). We’ve been fortunate to have supported several clients in developing and submitting draft protocols to CDRH, so have benefited from a “heads up” about many of the changes that are now embodied in this final guidance document. The review can be a great way to confirm that your approach to HF Validation will satisfy expectations.

If testing of labelling formed part of the validation study, the agency now require this to be conducted separately after simulated use testing.

The test protocol must provide a rationale for the extent of device use and the number of times that participants will use the device. This means frequency of device use must be considered; for devices that are used frequently and have a learning curve that requires repeated use to establish reasonable proficiency, testing should allow participants to use the device multiple times, if appropriate.

Data collection should include capture of observational and knowledge task data and should be supplemented with subjective data through the use of de-brief interviews with the participants after the use scenarios are completed. The purpose of the de-brief is to assist with understanding of possible root cause of use errors detected during testing.

Study location

It has long been understood that there is an expectation for the validation testing to be performed in the US. It is an expectation that hasn’t been removed, as the results of studies performed in other countries may be skewed by differences between healthcare practices, cultural and language differences. However, the agency are open to considering exceptions, on a case by case basis, where there is a sound rationale that addresses the differences. In any case, as you might expect, the device, labelling and training must be those that would be used in the US. So, in the vast majority of cases, it is likely that testing in the US would be more straightforward.

When is the guidance effective?

You will have some time to assimilate the requirements…
A whole 5 weeks, as they go live on 3 April 2016.
How will these changes will affect your product development?
What impact they will have upon your Human Factors Engineering programme?
Get in touch today to discuss how you can best navigate the changes and emerge with a Human Factors programme that is “just right”.

Share this with your network