Author Archives: Fiona Theobald

The forgotten twin?

Human Factors and Usability are “hot topics” in the medical device world (and increasingly for drug/device combination products too), yet the concept and the disciplines are not new. Designers and Developers have, at some level, been considering the user for years. But, it’s not just about “User Centred Design”.

A toe in the water with Human Factors

When I started in Device Evaluation, more years ago than I care to remember, we thought we viewed the device designs from a user perspective. I conducted Failure Mode and Effect Analyses (FMEAs) looking at all the ways users could destroy or misuse our devices. We even did a lot of user trials, though they were mostly with colleagues. My first thought, when I came across “Usability” back then, was that it’s the stuff we did when we tested the instructions with others in the development team.

Device Evaluation and Human Factors

How perspectives change. Across more than 15 years in Device Evaluation , I spent many happy months destroying devices (known as robustness testing) in a materials testing lab. Looking back, Human Factors at that time seemed the forgotten twin sister of device testing. It was the element often overlooked or dismissed in favour of purely physical testing.

Sometimes, I started testing a device in a lab, only to discover that I used it incorrectly and had to start over. At the time I put these lapses down to being forgetful, not concentrating properly or that the device was “just annoying”.

In the intervening years, I’ve studied and practised Human Factors in depth. You might be expecting me to recount the story of my turning point. Yet I had no sudden epiphany, no ray of sunlight breaking through the clouds. Rather I had a growing appreciation of the rich variation in how people think and behave.

Study has armed me with a more extensive arsenal of tools with which to understand users better. Experience has taught me that we ask a lot of people, when they get hold of these devices. Even tasks as simple as removing a device cap can be tricky. In fact cap fit is one of the hardest things to get right with many devices, particularly combination products.

The dangers of focussing on just Device Evaluation

That reminds me of a combination drug / device project I worked on a few years ago. I spent many hours performing cap fit tests, passing and failing batches based on the axial force required to pull the caps off. Part way through the tests I observed a user study where most people applied a sideways bending action to pull the cap off. The sideways bend weakened the device, causing it to fall apart shortly after. I had to question the point of all that axial testing… so much for fine tuning the cap design for the axial pull apart force!

Performing user studies, or getting “real” user thoughts, can make a world of difference to what you classify as being critical to design. Yes, the axial pull apart force was important to cap removal, but critical to creating a robust design was how the device was really used.

Why Human Factors and Device Evaluation are closely related

old twin sistersYou can perhaps see why these two disciplines are twins. Often, one of the two seems to be ignored or forgotten by development teams. Perhaps that’s a result of the twins being very different in nature, as siblings often are. On the one hand, you have a discipline that is very method and results data driven, mechanical and “hands on”. On the other hand you have a discipline that’s focussed on thoughts, actions and reactions of uncontrollable test participants.

Development teams seem to have a natural preference for one or the other, a comfortableness that fits with their collective learning style. Similar to our development as human beings, recognising the potential weakness or blind spot is the first (and biggest) step to beefing it up, to get the balance right.

What happens if you focus too much on one twin?

Consider what happens when you do as much physical lab testing as you like, but miss something obvious like how people really remove a device cap. Then, it’s back to the drawing board for the design, with all the delays, costs and lost profits that entails.

Consider what happens when you do lots of Usability testing with your device design, get great results from the final design iteration, but the device falls apart or stops working as expected when subjected to repeated use. Again it’s back to the drawing board to improve the robustness of the design, with delays, cost increases and lost opportunities for profit.

It can be hard to recognise an imbalance between the twins and address it.

You may already be thinking about getting in touch with us to check if you’re treating both siblings equally and how to achieve balance. Pick up the phone or email us today.

Closing the gap between EU and US Human Factors requirements

Medical device manufacturers can have a tough time gaining approval of new devices in the US, due to the increased scrutiny of the usability aspects of device design. To some the EU has been seen as a little more “relaxed” when it comes to testing and documenting device usability; the usability engineering process for medical devices is implied and not demanded as part of regulatory submission.

But now, the playing field is levelling out.  The standard for Usability/Human Factors ( IEC 62366) was recently updated to IEC 62366-1: 2015.  May seem like a subtle difference, yet the content change is significant if you’re developing either a combination product or medical device.

IEC 62366 was closely tied with the Risk Management standard, ISO 14971: 2007 and crammed full of examples of processes and documentation. Until this year, 62366 only required that a Usability Engineering File was created and that usability studies were carried out to verify and validate the safety of the medical device design. Now, however, the usability engineering file must contain results of both formative and summative testing,  Summative testing is used to verify and validate the medical device design (IEC 62366-1:2015 subclause 5.9).

There is also now an expectation that summative testing will find use errors that must be analysed, to identify the root cause and determine whether their severity warrants further risk control measures.

Ambiguity over setting study acceptance criteria has been removed at a stroke. Gone are the days when the Human Factors standard used to imply that, for example, a 90% success rate in correctly using the device was acceptable. The focus has now clearly shifted to examining what happened to the 10% who failed to use the device correctly and the consequences for user safety.

All these changes to study design and acceptance criteria requirements will bring EU activities closer to the FDA requirements.

As Europe tightens the usability requirements of medical devices, the only remaining difference between the FDA and the EU is; at what point will the regulators ask to see your Usability Engineering process; pre or post regulatory dossier submission?

If you need help and support or would like a quick chat about your usability engineering programme, get in touch to learn how we can help you improve the usability of your product.

Try this to set clear, straightforward goals, so you develop the right product

I had a flash of inspiration during lunch with a colleague last Friday.

Our conversation included how difficult it seems to be to set a clear, concise statement of the objective for developing a device (or indeed any product). The conversation drifted from there to the most memorable presentations we’d experienced.

My eureka moment; the power of a story to anchor and clearly describe the objective. I’d like to share with you an experience that I still vividly recall.

There was an AAMI/FDA meeting in early 2012. On the first afternoon, I listened to a series of presentations around the perhaps dry topic of “device clock synchronisation challenges”. After hearing several speakers talk about technicalities, standards and protocols, I still didn’t really appreciate what the pressing issue was. Then James Fackler, MD (The Johns Hopkins University School of Medicine) shared his experience of working in a paediatric ICU…ICU device array

The first slide was of a large bank of very similar looking devices in an ICU.

The doctor was faced with banks of instruments all humming and bleeping away, all alerting them to their patient’s bodily functions.

Slide two had a graph of an output versus time from one of the instruments.

The chart was explained, then a second graph was added, followed by a third, fourth and fifth; each with a trace from another device in the ICU, potentially covering the same 30 second time period.

The MD then talked about the blips on the traces and what they meant to the healthcare professional.

Then came the shock.

The blips on each of the charts showed failure of a different bodily function. Crucially, the relative timings of these blips meant a different surgical procedure may be needed to save the patient, a child.

It was like a smack on the forehead!

I understood how important the chart accuracy was to the doctor. A decision how to proceed has to be made within that 30 second window, so device clock synchronisation was critical to presenting the correct sequence of blips that pinpointed the next steps. The doctor is wholly reliant on data from all the devices being accurately displayed, at the right time points, in order to select the appropriate treatment.

Device clock synchronisation suddenly became a matter of life and death.

Encapsulating the problem in a story, clearly presented what needed to be solved and why it mattered, in a straightforward and profound way.

As an aside, although there were no hand-outs from this session, it’s the one I recall most readily from the meeting.

Maybe, as you read this post, you reached the same realisation about the life saving importance of device time synchronisation. How powerful it would be if you described your development goals in terms of what it will mean for the eventual user; described them so that everyone understands exactly the imperative you’re working to address.

Zen and the Art of Design Controls

There are days when I think “is it me?” Especially when it comes to appreciating Quality Systems. I confess I do like logic and I even get a buzz when I see a logical progression in project deliverables, moving from A to B to C and so on. It’s wonderful, having links and traceability.

I must make it clear I have never been part of a Quality Department. I have been involved in medical device development for many years and lived through, what I now think of as the unenlightened times (although immensely fun), when we defined our own ways of working, prior to the introduction of Design Controls and ISO 13485; having standards and guidance is now quite comforting for me. This probably explains why I find it distressing when I come across people who do their level best to ignore or get round the logical order laid out in Design Controls.

I struggle to grasp why it is faster to go from A to C and then back to B and then try to jump to D. Worse Tortoise and Hareyet are those who think they can start at C and miss out A & B, until they reach stage F. I have experienced this in various guises and I don’t think I have ever seen a project delivered anywhere close to the original time lines when run like this, usually missing the mark by several years.

Hind sight is a wonderful thing and yes I have on occasion been known to fall back on it, having been carried along with the fast track idea. However, when it comes to building up a sound development package, skipping through stages invariably comes back to bite you if you choose to miss bits out.

Filling documentation gaps that should have been produced much earlier in a project is always fun.

No, I am lying, it really isn’t.

If you have ever had to do this you’ll know that you find out why your project hasn’t gone as swimmingly as you had hoped it would (back to that pesky hindsight thing). Here are three real life examples:

  • Producing the User Requirement Specification for a product when the product design is already decided and having the realisation that the wrong (unjustifiable) features have been added or that critical features are missing from the design is always a hoot.
  • Completing a design FMEA and dreaming up a raft of new and impossible to execute tests after all of the Verification testing has finished.
  • Finally a personal favourite, conducting usability studies with no idea; who the target audience is, what their requirements are or how the device is intended to work.

There has been much in the press lately about the brilliance of 3D printing and undeniably it is fantastic what can be achieved. But there is a flip side; 3D printing has made the possibility to jump around Design Control stages even easier. Designers can now generate what looks like a finished product before anyone else has even had the chance to think about what the problem is that the development team are trying to solve.

It is all too easy to focus on refining or “optimising” this nearly finished product. Contrast this with concepts presented as block models, sketches or simulations. There’s an inherent understanding that much work lies ahead to turn the best concept into a fully fledged, marketable device.

Design Controls CascadeThere are many graphics out there that try to explain Design Controls. The one thing they have in common is that; to get a good output you need a good input, to get a good input you need a good understanding of the problem you are solving.  For me following the flow of Design Controls takes away a lot of development pain and anguish of trying to guess what comes next.

Design Controls is basically a logical progression built from the years of knowledge of other people who have already suffered in development limbo and have found a clear, signposted way through.

“Begin with the end in mind”
Stephen Covey

Look out for our post later this month that may help you think about the point of each stage in Design Controls, enabling you to get the most out of each stage of development.