UK Perspective Regarding FDA Regulatory Requirements

A Linked-In colleague posted a link to this article. I read it and found it interesting enough to post the link and comment on it. It's by a UK publication and discusses the FDA regulatory process as it relates to Human Engineering requirements for device approval for commercialization.

Here's the link:
http://www.emdt.co.uk/daily-buzz/what-are-fda-usability-testing-requirements-device-approval?cid=nl.qmed01.20150325

In addition, I provide my own perspective on the article in the "Commentary" section below. I do not critique the article. I only attempt to expand on a few points from it.

But first, a brief summary of the article.

Article Summary

Medical errors have become an increasing concern of the FDA. I became interested in medical errors when I was a consultant at St. Jude Medical Cardiac Rhythm Division in Sylmar, CA. During my time at St. Jude (2009-2010), deaths by medical error were being reported as being 100,000 to 120,000 per year. Last year, I posted links to two articles that stated that deaths by medical errors could be closer to 400,000 per year. (http://medicalremoteprogramming.blogspot.com/2014/07/rip-death-by-medical-error-400000-year.html)

It has been noted by the FDA a large proportion of medical errors can be attributed to poorly designed medical device user interfaces. Since a fundamental mission of the FDA is increasing patient safety and reducing injuries and fatalities in the practice of medicine, the FDA has begun placing greater emphasis on improving the usability of medical device user interfaces.

This article provides measures that show the FDA's increasing emphasis on usability and human factors issues by showing the increasing frequency that companies seeking medical device clearance for the US market mention the terms "usability" and "human factors." Figure 1 from the article clearly shows the increasing usage of these terms in company filings.

The focus should be on the trends, not the absolute numbers because not all filing documents have been included in the count. But the trend clearly shows an increased emphasis by companies to increasingly use the terms "usability" and "human factors" in their filings with the FDA. The two figures that follow suggest the degree that companies have incorporated the FDA prescribed human factors engineering process and design guidance documentation.

The documents listed below are specifically targeted to defining and supporting the human factors engineering process and the development of the Human Engineer File that's included as part of a company's filing to the FDA.

• ISO 62366, Medical Devices - Application of Usability Engineering to Medical Devices
• AAMI / ANSI HE75:2009, Human Factors Engineering - Design of Medical Devices (General)

I'll discuss the documents above in greater detail and describe how they're intended to fit within the human factors engineering process when developing medical devices.

• IEC 60601-1-6 Medical electrical equipment - Part 1-6 General requirements for Safety - Collateral standard: Usability
• IEC 60601-1-8 Ed. 1, Medical Electrical Equipment - Part 1-8: General Requirements for Safety - Collateral Standard: Alarm Systems - Requirements, Tests and Guidance - General Requirements and Guidelines for Alarm Systems in Medical Equipment (General)

The two documents above are engineering standards. They're engineering specifications that medical devices must meet. They are technical and specific.

I show Figure 3 from the article before showing Figure 2. 

The increasing reference of 60601-1-8 is not surprising given the increased emphasis on safety. My real interest is in the significant increase in reference to ISO 62366. As mentioned, this is process standard the lays out how human factors engineering should be engaged to reduce "use errors." The emphasis in this standard is on the reduction of risk. Risk management is extremely well embedded in the medical device design and engineering process. It would seem that from a cultural perspective, ISO 62366 fits with the medical device engineering process. 

I want to contrast the dramatic, increasing references to ISO 62366 with the references to AAMI/ANSI HE75 shown in Figure 2 below.

References to AAMI/ANSI HE75 rise and fall from 2010 to 2013 instead of a steady upward trend that you see with ISO 62366 in Figure 3. I would like to emphasize that ISO 62366 and AAMI/ANSI HE75 should be considered as companion documents. (I'll expand on this in the Commentary section below.)

Commentary

The article does support the contention that the FDA and the companies it regulates are paying increasing attention to usability and human factors. That they're paying enough attention is another matter entirely. As new medical devices are introduced we should see two things. First, the use error rate for the newly introduced medical devices (once users have adapted to them) should decline in relationship to other similar devices currently in use. Second, we should see over time the number of per year of deaths and injuries from medical errors begin to decline. This will take time to detect.

Without a doubt, the push by the FDA to define a human engineering process in the design and testing of medical devices, and to press for testing under actual or simulated conditions is needed. In many ways the FDA is mirroring many of the processes that have already been adopted by the US Department of Defense (DoD) in the area of human engineering. Admittedly, the DoD doesn't always get it right, there is an understanding within the DoD that it is important ... life saving, battle-winning important ... to insure that those at the controls can do their jobs quickly, effectively and with as few errors as possible.  So from that standpoint, the FDA has adopted processes from programs that have proven effective. But the FDA has just passed the starting line. And much more will be required going forward.

ISO 62366 vs AAMI/ANSI HE75

As I mentioned earlier ISO 62366 and AAMI/ANSI HE75 should be consider complementary or companion documents. HE75 is a much larger document than 62366 and includes a significant amount of device design guidance and guidelines. 62366 is almost entirely a process document that's devoted to directing how to go about managing the research and development process of a medical device. In addition, the focus of 62366 is managing risks, risks in the realm of reducing use errors.

I found it interesting that references to HE75 were not increasing at the rate as references to 62366. I would have expected Figures 2 and 3 to have a similar appearance with respect to 62366 and HE75 in large part because the documents significantly overlap. In fact I might have reasonably expected references to HE75 to outpace 62366 because HE75 includes design specific guidelines in addition.

One possible reason for references to HE75 not being referenced in the same accelerated way as HE75 may have to do with the fact that the European Union has not adopted HE75, so it's required for medical devices that will be marketed in the EU (CE).  (I am currently unaware of the regulatory requirements of other countries on this matter.) Medical device companies are international companies and the documents that they file in one country are generally the same in each country. Thus since the EU hasn't adopted HE75, references to HE75 and HE75's use as a foundational process and design document may be less.

DESIGN RATIONALE

I'm not sure that this is true at this point in time, but I am certain that the following will be true going forward at some time in the future. I believe that the FDA will hold companies to account for their user interface designs. I believe that the FDA will demand that companies clearly define how they came up with their user interface designs and that those designs are well-grounded in empirical evidence.

This is what I mean ... the FDA will demand that the design choices ... these include: controls, placement of controls, number of controls, actions performed by controls, the way the control responds, methods for interacting with the device (e. g., touch screen, buttons, mouse), size of the display, etc. ... for medical device user interfaces must be grounded in empirical data.

Commercial websites are often designed by graphic artists. Often times the design of webpages reflect the artist's aesthetic sensibilities. Layout appear they way that they do because they look good.

I believe that the FDA will require that user interface designs for medical devices have an empirically grounded design rationale. Companies will be required to point to specific research finding to justify the design and the design choices that they made. Furthermore, as the design of the user interface evolves with each iteration of testing, the FDA will require that changes to the design be based on research findings.

Finally, I believe that soon if it is not occurring already, that the FDA will require:

1. That companies submit documentation to show in detail the full evolutionary design process beginning from product inception, including ...
2. Detailed pre-design research ... population(s), method(s), research questions and rationale, etc ... as well as the findings and what they suggest for the design of the user interface
3. A design that includes with a full discussion of the design rationale ... why was it designed the way it was ... 
4. A detailed description of the evolution of the design that include full and clear justification(s) for each change in the design ... and require that changes be grounded empirical data 
5. A full description of pre-commercialization testing process and method ... with a clear justification for why this testing meets FDA testing requirements
6. And a complete and clear analysis of the testing data.

What I'm suggesting above is that the process of designing and testing a medical device user interface should be more than going through the prescribed steps, collecting the data, doing the tests, etc. There should be a clear thread that ties all the steps together. When in a subsequent step, one should be able to point back to the previous steps for the rationale to explain why the user interface was designed to appear and operate the way it does ... to this point.

As near as I can tell, what I described above is rigorous than is currently required by the FDA. However, I believe that it would be in any company's best interest to follow what I've suggested because there may come a time when the FDA's enforcement becomes more rigorous. 

Another reason may be lawsuits. If a company can show that it went beyond the FDA's regulatory requirements at the time, those suing would likely have less of a chance of collecting damages. And if damages were awarded, they may likely be lower. Also, if the company went beyond the FDA requirements, it would be likely that there would be fewer people injured and that should lower damages.

FINALLY

This article has been a springboard for me to discuss a number of topics related to human engineering for medical devices user interfaces. This topic will remain a central part of this blog. I'll return this within a week or two, and discuss in depth other topics related to the human engineering process for medical device user interfaces. 

How This Blog Got Going: MRI Safe and Conditional Pacemakers, Reprise

I have decided to return to the thing that I was working on when I started this blog ... an MRI conditional pacemaker. Specifically, an MRI conditional pacemaker for St. Jude Medical. At the time I was Lead Human Engineering Clinical Systems Engineer on this project. Before I go any further I would like to distinguish between MRI conditional and MRI safe devices. It is important to distinguish between the two.

MRI Conditional v. MRI Safe

Having an MRI safe implanted cardiac device is the ideal situation. If the cardiac device is MRI safe, it means that a device patient can be "popped" into an MRI without any changes to the device. For the patient it's just like the person does not have an implanted device. The only difference is that the resulting imagery from the MRI around the device may not be as good if the person did not have an implanted device. 

An MRI conditional device presents some significant procedural challenges to all those involved. If a person has an MRI conditional device, certain conditions must be met before the device patient is allowed to enter the MRI. When I was working at St. Jude Medical, changes in the settings that operate the device are required before the patient enters the MRI. Once scanning is complete, the settings need to be changed back to their normal, operational settings.

As of publication of this article, only one medical device company has a commercially available MRI safe pacemaker, Biotronik. St. Jude Medical and Medtronic have commercially available MRI conditional devices. 

When I was work at St. Jude, the only cardiac device being engineered to permit patients to have MRI scans were pacemakers. At the time ICDs and CRTs were not considered for MRI compatibility. However, apparently, Biotronik has developed an MRI conditional ICD that is commercially available ... at least in Europe.

There are other issues regarding MRI compatibility such as whether there are limits on the area that can be scanned a cardiac device patient ... something other than a full body scan. The allowable limits on how much can be scanned are continually in flux. But this particularly issue does not have anything to with the story I want to tell.

My Experience with the MRI Conditional Project

The St. Jude Medical MRI conditional pacemaker was engineered to enable patients to undergo an MRI scan. To insure that pacemaker patients would not be harmed by the scan required that the operating settings on the pacemaker be adjusted. (To make a long story short ... a change in the setting needed to make sure that the sensing lead to heart be turned off. The pacemaker could be changed to constant pace or turned off entirely if the patient is not pacemaker dependent ... as most pacemaker patients are.)

So the major problem in this entire issue was in regards to how to change the settings on the device? Who would do it, how would it be done, what would the settings be? Essentially three basic approaches were considered:

1. Have the patient's cardiac physician or cardiac nurse go to the MRI center, lugging their device programmer with them, change the settings on the patient's device to those that are MRI compatible, wait for the scan to complete, reset the settings to normal and examine the patient to insure that the patient is OK.
2. Have the settings changed remotely. The patient is at the MRI center, the cardiac professional is in the office, at the hospital or at home. This is known as "remote programming."  At the time this was something that the FDA did not allow. Using remote programming, the patient's device communicates wireless to a pacemaker communicator located at the MRI center. The cardiac professional sees a 30 second rhythm strip before setting the patient's device to the MRI settings and sees another 30 second rhythm strip after the changes have been made. (Just like an onsite cardiac professional would do.) The patient undergoes the scan. During that time, the professional can perform other tasks. Once the scan is complete, the cardiac profession changes the pacemaker settings back to normal and sees the before and after rhythm strips. 
3. The pacemaker is programmed with two settings by the cardiac professional using the programmer. The first set of settings define the normal operation of the pacemaker. The second set are the MRI settings: that is, the settings of the pacemaker when the patient undergoes an MRI scan.  When the pacemaker patient goes to the MRI center, the MRI tech takes a wand (that's best way I can describe it.) and changes the settings from normal to MRI. Once the patient completes the MRI scan, the MRI tech uses the wand to change the patient's setting back to normal. 

I became quickly apparent that cardiac professionals had no interest in option 1. As it turned out St. Jude Medical chose the third approach. 

When the third approach was described, I had numerous objections ... mostly related to the device that would change the setting on the pacemaker. Thankfully, there have been substantial changes and upgrades made to the wand. However, I wanted to purse option 2, remote programming. And the desire to purse option 2 inspired me to start this blog ... hence the title Medical Monitoring & Remote Programming.

Wherefore Remote Programming?

Most physicians showed some hesitancy when it came to adopting remote programming. They saw it as unproven ... and they were right, it was (and so far as I know still is) unproven and still not acceptable to the FDA. However, many if not most were intrigued by the idea and thought that the technology should be pursued. Many clearly saw the potential value of the technology, the value of being able to monitor patients remotely with the potential ability to change cardiac device settings without the patient being in the office could be a revolution in patient care ... not only for people with chronic conditions like heart problems, diabetes or neurological problems that involve implanted devices, but potentially everyone. And it need not involve the need for implanted or wearable devices. We'll explore this in later postings.

Hacking Grandpa's ICD: Why do it?

Background

I am part of another professional discussion group with an interest in Medical Data, System and Device security.  One of the topics was whether medical devices are a likely target for cyber-attacks.  I made a contribution to the discussion and stated that I believed that although unlikely, I thought that medical devices will eventually be targets of cyber-attacks.  But putting data security measures into medical devices is at odds with the directions that the medical device industry wants to take its product lines.  The trends are for smaller and less power-hungry devices.  Adding data security measures could increase power demands, increase battery sizes and thus increase device size.  Nevertheless, I believe that starting the process of putting data security measures into the medical devices has merit.

I received a well-reasoned response that hacking medical devices was highly unlikely and research funding on security measures for medical devices would be money best spent elsewhere.  That response started a thought process to develop a threat scenario to address his points.

I reviewed my earlier article on "hacking medical devices," http://medicalremoteprogramming.blogspot.com/2010/04/how-to-hack-grandpas-icd-reprise.html.  I revisited the paragraph in my regarding the motivation for hacking a medical device, an extortion scheme. 

When I wrote that article, I did not have any particular scheme in mind.  It was speculation based more on current trends.  Furthermore, I did not other motivations as particularly viable - data theft, not much money or value in stealing someone's implant data or killing a specific person, there are easier ways to do this although it might make a good murder mystery.

I did come up with a scenario, and when I did, it was chilling.





The Threat Scenario

First, as I had previously suggested, the motivation for hacking medical devices would be extortion.  The target of the extortion would be the medical device companies.  Before getting into the specifics of the extortion scenario requires that you understand some of the technologies and devices involved.

The wireless communications of interest occurs between a "base station" and a wirelessly enabled implanted device as shown in the figure below.

The base station need not be at a permanent location, but could be a mobile device (such as with the Biotronik Home Monitoring system).  The base station in turn communicates with a large enterprise server system operated by the medical device company.


The two systems communicate use wireless or radio communication.  For example, St. Jude Medical uses the MICS band - a band designed by the FCC for medical devices in the range of 400Mhz.  To insure that battery usage for communications is minimal, the maximum effective range between is stated as 3 meters.  (However, I have seen a clear connection established at greater 3 meters.)  


In general, the implant sends telemetry data collected it has collected to the base station.  The base station sends operating parameters to the implant.  Changing the operating parameters of the medical device is know as reprogramming the device and define how the implant operates and the way the implant exerts control over the organ to which it is connected.


Device Dialogue of Interest to Hackers

As you probably have guessed, the dialogue of interest to those with criminal intent is the one between the base station and the device.  The "trick" is to build a device that looks like a legitimate base station to the medical device.  This means that the bogus device will have to authenticate itself with the medical device, transmit and receive signals that the device can interpret.  In an earlier article (http://medicalremoteprogramming.blogspot.com/2010/03/how-to-hack-grandpas-icd.html), I discussed an IEEE article (http://uwnews.org/relatedcontent/2008/March/rc_parentID40358_thisID40398.pdf**) where the authors had constructed a device that performed a successful spoofing attack on a wireless Medtronic ICD. So, based on the article, we know it can be done.  However, based on the IEEE article, we know that it was done at distance of 5 cm.  This was aptly pointed out in a comment on my "How to Hack Grandpa's ICD" article.


Could a Spoofing/Reprogramming Attack be Successful from Greater than 5 cm or Greater than 3 meters?


I believe the answer to the question posed above is "yes."  Consider the following lines of reasoning ...

1. As I had mentioned earlier, I know that base stations and medical devices communicate at distances of 3 meters and can communicates greater distances.  The limitation is power.  Another limitation is the quality of the antenna in the base station.  The communication distance could be increased with improvements in the antenna and received signal amplification. 
2. The spoofing/reprogramming attack device could be constructed to transmit at significantly greater power levels than current base station.  (Remember, this is something built by a criminal enterprise.  They need not abide by rules set by the FCC.)  Furthermore, a limited number, maybe as few as one or two, of these systems need be constructed.  I shall explain why later.
3. A base station can be reverse-engineered.  Base stations can be easily obtained by a variety of means.  Medical devices can be stolen from hospitals.  Documentation about the communication between the medical device and the base station can be obtained.

Thus, I believe the possibility exists that a device that emulates a base station and could successfully perform a spoof/reprogramming attack from a significant distance from the target is possible.  The question is, what is to be gained from such an attack?


Attack Motivations


Extortion: Earlier I mentioned that in an other article, I suggested that the motivation would be extortion: money, and lots of it.  I think the demands would likely be in the millions of US dollars.

In this scenario, the criminal organization would contact the medical device companies and threaten to attack their medical device patients.  The criminal organization might send device designs to substantiate their claims of the ability to injure or kill device patients and/or send the targeted company with news reports sudden unexplained changes in medical devices that have caused injuries or deaths in device patients.


Market Manipulation: Another strategy would be as a means to manipulate the stock prices of medical device companies - through short-selling the stock.  In this scenario the criminal organization will create a few base station spoofing/reprogramming systems. Market manipulation such as placing the value of the stock at risk could be a part of the extortion scheme.




Book of Interest: Hacking Wall Street: Attacks And Countermeasures (Volume 2)


In another article I'll discuss how someone might undertake an attack.




** Halperin, D, Heydt-Benjamin, T., Ransford, B., Clark, S., Defend, B., Morgan, W., Fu, K., Kohno, T., Maisel, W. Pacemakers and Implantable Cardiac Defibrillators: Software Radio Attacks and Zero-Power Defenses, IEEE Symposium on Security and Privacy, 2008, pp 1-14.

Know Thy Target Population

Background

Members of the project core team including myself were sitting around a meeting table at St. Jude Medical.  We had received a sample of work from a well-known human factors and design consulting company.  It was a detailed design of a user interface of a web-based system that the consulting company had designed.  (I withhold the nature of the system because some of you may know this consulting company and have received the same sample system.  I want maintain the anonymity of this company.)

Most of the members of the core team were impressed by the design and the level of detail provided in the sample of work.  I was not impressed.  Why?  There was nothing in the design that tied back to the target population the system would serve, nor was their any reference to any research performed that justified the design or the services that would provide its users.  In other words, I had no way to evaluate whether the design was good or bad, whether it suited the user needs or not.  Furthermore, there was no information provided by the consulting company regarding the target population, its needs, skill-level, etc.


If I understand the objectives of HE75 correctly, then medical device companies will be required to demonstrate through research - it is unclear whether the research required will involve the need for empirical research or a literature search.  However, companies will be required to demonstrate a clear understanding of the targeted population and the use environment of that target population for whom their products and services are intended for use.  The consulting company who provided St. Jude Medical with work samples failed to demonstrate any knowledge of the targeted population.

Know Thy Target Population

And know the use environment of thy target population: objectives, distractions, dangers, skills, needs, etc.

There are few things that rankle a human factors professional more than an ascetically pleasing design that fails to take in account the population of the actual users.  I will not discuss any specific examples in this article.  However, you can look around the Web, around your home or office, or a hospital or clinic and see all kinds of products and displays that were designed without regard to the target user population.  Companies continue to build products with user interfaces ill-suited to the target populations.  Why?

Designing research, collecting and analyzing data and turning the analysis into appropriate products and in particular, well-suited user interfaces takes time and money.  In highly competitive environments, time can be more of a concern than money.  For companies that have adopted a "fast follower" business model, time often looms larger than money when planning and developing products, and designing user interfaces.  Products based on a fast follower approach can often be successful if the product or product upgrade does not require a user interface.  Development is nothing more than the implementation of a proven algorithm.  However, when there is a user interface, the lack of attention to understanding the target population can become painfully obvious once the product or service is introduced. 

When a company uses a "fast follower" model or does not want to devote the time, effort and money to research, the temptation can be to rely on opinion leaders, other companies' products, best guesses, or "expert" opinions.  These approaches often bring fast outcomes.  Furthermore, I have noticed that often times decision-makers believe that they can determine what is and what is not a good user interface - that a good user interface design is something that anyone can assess. 

I believe that some of the faith that decision-makers have placed in themselves comes from their experience with consumer products, such as the Apple's iPhone.  The iPhone is an combination of excellent physical design ascetics and small, touch-screen user interface design.  What many people forget is that Apple devoted copious amounts of time, money, expertise to research and development of the physical design and the user interface of the iPhone.

The iPhone was designed with the general public in mind.  Apple may have targeted the iPhone to a more youthful population, however, I have noticed that many, many older people pulling out their iPhones.  I have also noted that there are numerous medical applications targeted to iPhones, applications that would be of interest to older people.  The iPhone may have been targeted to a more youthful population, but its touchscreen interface with its large, wide buttons, high contrast and high resolution screen enables elderly users to access its capabilities.

When user interfaces are targeted to a specific population, it becomes imperative that the characteristics, needs, qualities, etc. of the target population be well-known and understood.  The user interfaces introduced to that population will likely be tied to a specific set of tasks with clear objective within a specified environment.  The use environment maybe more complex and stressful than the use environment of a consumer product.  Furthermore, and this is particularly true of many medical devices, the consequences of making an error can be significantly more harmful than any consumer product.

For decades, the Department of Defense has pressed its contractors to follow standards and demonstrate usability in the use environment.  The first step in that process is understanding the target population and the use environment.  Given the accumulating evidence that medical errors kill more people per year than automobile accidents and war, I believe that the FDA will take strong steps such as the steps taken by the Departments of Defense and Transportation.  Judgment will no longer substitute for data and analysis.  The evidence that I have seen is that HE75 and HE74 will provide significant guidance towards directing companies towards gaining a full understanding of the target population and their use environment.

One thing to note, target populations and use environments are dynamic, not static.  Continuing research is essential.

Next time: Now that you have an understanding of your target population and the use environment, what do you do next?  I shall come back to specific techniques for research a target population and their use environment in a later article.  However, I want to provide a process overview before diving into specifics and specific techniques.