Tag Archives: validation

Three ways to smooth your path for device development

Medical Device Development routemap v5.0 narrow viewSpring of 2013 still seems recent, so I was surprised to realise that the route map for medical device development will shortly be 3 years old. If it were a child in the UK, we would be able to get 15 hours free childcare a week from this September.

Children learn and grow at an alarming rate, especially when you’re looking back over what seems like a very short time and realise that the baby you remember holding in your arms “yesterday” is now a teenager.

The route map too is growing up fast. It’s being used across the world, from Australia, through Asia to Europe and on to North America. Users have fed back great experiences. Uses ranging from a desktop aide mémoire, a teaching aid, through to a planning and design review tool!

There comes a time when a child learns new skills, and naturally integrates them into their daily lives and toolkit.

We’re proud to announce the launch of the route map for medical device development 5.0 today. Numerous user suggestions have been incorporated in this new version, along with two new tracks; Intellectual Property and Market Research.

The route map continues to help you in three ways, it;

  1. Describes the journey you’ll embark upon, from a device idea through to launch.
  2. Sets out the main activities for each of 9 disciplines, through four stages of product development.
  3. Identifies the points where there are major interactions between disciplines.

And here it is…

Click to open a pdf version you can download

Click to open a pdf version you can download

You may already see how this picture will be helpful, whether you’re working in medical devices, combination products, pharmaceuticals, aerospace, defence, electronics…..

Send me your thoughts on how you can use the route map, because having a straightforward, independent view is always valuable.

FDA Finalises Human Factors Guidance – summative testing – Part 3

In this series of posts, we’re taking a look at the recently published final version of CDRH’s guidance “Applying Human Factors and Usability Engineering to Medical Devices”.
We have already taken a look at the first three in this list of the key topics covered by the Center:

  • Human Factors Engineering processFDA HF guidance FINAL
  • Risk Management and Human Factors
  • Design Verification
  • Summative Human Factors testing
  • Changes to products already on the market
  • Human Factors Engineering Summary report

In part 3, we’ll look at how the topic of Human Factors testing (both formative and summative) may impact what you’re either already doing, or plan to do.

Formative Human Factors testing

Look upon this stage of testing as providing a toolkit to identify potential use-related hazards and hazardous situations. The agency emphasises the value of testing with participants who are representative of your intended users, as this yields early data on how your eventual users cope (or don’t) when using your prototype device.

The agency recognise that formative evaluations provide information about;

  • Design of the user interface (those parts of the device that are manipulated by the user, provide them with information or indicate function)
  • The effectiveness of features or changes implemented to reduce or eliminate use errors (otherwise known as mitigations)
  • The the need for training of users and inform the design of training materials
  • The design and content of labelling such as the Instructions For Use, product labelling, packaging.
  • The content and design of your summative testing.

In the guidance, the Center talk about the value of effective formative testing in preventing very costly errors in the design of your validation test protocol that render it just another formative study.

In a departure from previous guidance, there is information on the expected content of formative test protocols, which creates a coherent and building picture that results in the information now expected for your HF validation test protocol. It’s very much a case of using formative studies as building blocks that provide a robust foundation for your validation test.

Summative Human Factors testing

Summative testing is still in place. However, testing that demonstrates the “user requirements” or User Needs have been met will henceforth be referred to as “Human Factors Validation Testing”. One could perhaps see this as coming into line with the philosophy applied to validation of processes, test methods, production lines, etc

No longer does guidance expect you to just include critical tasks within your test design. Rather, the testing needs to be comprehensive, so that the results can be related to actual use.

Test participants

If your device is intended to treat patients with a condition that can cause them to have functional limitations, these conditions must be represented during both the formative evaluations and your HFE validation study. If you choose not to design your device to accommodate needs of those likely to use your device, then labelling should clearly explain the capabilities that users must have to use the device safely and effectively. You should, therefore, ensure that you’ve understood your Users’ needs thoroughly and scoped out the Usability Specification appropriately to know what capabilities/limitations are to be included in study participant groups.

Participant Training

If you anticipate that most users of your device would receive minimal or no training, the Center now expect that participants in your validation study do not receive training.

A period of time between training and participation in your validation study should be built in to the study design. How long that is will vary, depending upon what would happen in real life and the impact of training decay on the use-related risks that you’ve identified for your device. The important thing is to consider this topic and document your rationale for the chosen approach.

Study protocol

Manufacturers are now formally encouraged to submit draft HF Validation Test protocols for agency review, to ensure that the methods you plan to use are acceptable (via the pre-submission programme). We’ve been fortunate to have supported several clients in developing and submitting draft protocols to CDRH, so have benefited from a “heads up” about many of the changes that are now embodied in this final guidance document. The review can be a great way to confirm that your approach to HF Validation will satisfy expectations.

If testing of labelling formed part of the validation study, the agency now require this to be conducted separately after simulated use testing.

The test protocol must provide a rationale for the extent of device use and the number of times that participants will use the device. This means frequency of device use must be considered; for devices that are used frequently and have a learning curve that requires repeated use to establish reasonable proficiency, testing should allow participants to use the device multiple times, if appropriate.

Data collection should include capture of observational and knowledge task data and should be supplemented with subjective data through the use of de-brief interviews with the participants after the use scenarios are completed. The purpose of the de-brief is to assist with understanding of possible root cause of use errors detected during testing.

Study location

It has long been understood that there is an expectation for the validation testing to be performed in the US. It is an expectation that hasn’t been removed, as the results of studies performed in other countries may be skewed by differences between healthcare practices, cultural and language differences. However, the agency are open to considering exceptions, on a case by case basis, where there is a sound rationale that addresses the differences. In any case, as you might expect, the device, labelling and training must be those that would be used in the US. So, in the vast majority of cases, it is likely that testing in the US would be more straightforward.

When is the guidance effective?

You will have some time to assimilate the requirements…
A whole 5 weeks, as they go live on 3 April 2016.
How will these changes will affect your product development?
What impact they will have upon your Human Factors Engineering programme?
Get in touch today to discuss how you can best navigate the changes and emerge with a Human Factors programme that is “just right”.

Closing the gap between EU and US Human Factors requirements

Medical device manufacturers can have a tough time gaining approval of new devices in the US, due to the increased scrutiny of the usability aspects of device design. To some the EU has been seen as a little more “relaxed” when it comes to testing and documenting device usability; the usability engineering process for medical devices is implied and not demanded as part of regulatory submission.

But now, the playing field is levelling out.  The standard for Usability/Human Factors ( IEC 62366) was recently updated to IEC 62366-1: 2015.  May seem like a subtle difference, yet the content change is significant if you’re developing either a combination product or medical device.

IEC 62366 was closely tied with the Risk Management standard, ISO 14971: 2007 and crammed full of examples of processes and documentation. Until this year, 62366 only required that a Usability Engineering File was created and that usability studies were carried out to verify and validate the safety of the medical device design. Now, however, the usability engineering file must contain results of both formative and summative testing,  Summative testing is used to verify and validate the medical device design (IEC 62366-1:2015 subclause 5.9).

There is also now an expectation that summative testing will find use errors that must be analysed, to identify the root cause and determine whether their severity warrants further risk control measures.

Ambiguity over setting study acceptance criteria has been removed at a stroke. Gone are the days when the Human Factors standard used to imply that, for example, a 90% success rate in correctly using the device was acceptable. The focus has now clearly shifted to examining what happened to the 10% who failed to use the device correctly and the consequences for user safety.

All these changes to study design and acceptance criteria requirements will bring EU activities closer to the FDA requirements.

As Europe tightens the usability requirements of medical devices, the only remaining difference between the FDA and the EU is; at what point will the regulators ask to see your Usability Engineering process; pre or post regulatory dossier submission?

If you need help and support or would like a quick chat about your usability engineering programme, get in touch to learn how we can help you improve the usability of your product.

Base jumper or fell runner?

International standards such as ISO 62366, ISO 14971 compliance gurus and journals all talk about taking a risk based approach to elements of your development project.

As with many things in life, the strategy you choose to achieve this becomes a matter of how much risk you are prepared to live with;

  • How comfortable are you with the risk of meeting a costly surprise if you choose to test late in development?
  • How much risk are you prepared to pass onto your customers, your users?

Lets take a look at two strategies for a medical device development project.

Base Jumping

A high risk strategyThe dare devil approach. It’s a high risk strategy that is often seen as the fast way of getting down the mountain.

For this strategy to succeed, the device manufacturer has to be super confident in their design, either because it is very similar either a design they have already done or it is very similar to what’s already on the market. The design will also use existing production and assembly processes, without modification.

In this scenario most device testing and process validation activity is done at the last possible stage, when the design is frozen and the company is ramping up for launch.
Sometimes, testing is even done after launch when the regulators ask for the documented evidence.

To take on this strategy you have to have a lot of confidence that you know how to repeatably and reliably manufacture your product, your customers (and ultimate users) know how to work your device as well as you do.

After all, “Its not the fall that kills you”.

Fell Running

A more measured approachThis approach is often perceived as slowing down a fast track project. Yet, the strategy includes the right development activities, at the right times throughout the various stages of product development.

It’s the more conventional way of getting down a mountain.

“The sooner [you] get involved, the quicker, cheaper development is” Vicki R Lewis, AAMI/FDA summit 2013.

In reality, adding development activities to a project at the right time should not impact on time-lines, as activities can run in parallel. Indeed, these activities will greatly reduce the likelihood of expensive re-designs or fire-fighting as you get closer to scale-up and launch preparation.

Of course, development activities need to be budgeted for, but;

  • How much expensive is unfreezing a design to start developing new mould tools?
  • How much more expensive is a product recall because of a foreseeable use error?

Taking this approach should not end messily, as you descend your mountain.

Which of these two strikes a chord with you, feels like the usual one for development? Perhaps you can see the differences in likely outcomes from the two strategies.