Supplement to Part IV Submissions to FDA: Risk Management and Use Errors

Additional Thoughts:

As part of your risk analysis and use error identification, you should identify the circumstances or origin of the reported use error. 

• Origins of reports of use errors, in order of likely relevance:

1. Field reported use errors: These require special attention because they were discovered while in use by users under actual conditions. That's the reason why field reported use errors are the most important.They deserve special consideration, particularly if the use error was responsible for any harm and that the use error occurs significantly more often than originally predicted. Consider performing a root cause analysis to determine why the use error occurred. Most especially consider the use or environmental conditions and who made the use error to be factored into why the use error was made. Be sure to determine which assumptions regarding use, conditions for use and predicted user characteristics were violated. 
2. Errors reported from empirical studies: These are use errors that have been observed under laboratory or other kinds of testing conditions defined by researchers. Furthermore, these use errors are from members of the expected user population(s) who have the expected requisite level of education and training. Thus in testing sessions the conditions of use including the environment have been structured and manipulated by the researchers. The results of the research and the use errors detected may be valid, but narrow in scope in both situations of use, the use environment, actual user characteristics including education and training, etc. 
3. Analysis based:
• Scenarios: Scenarios are set of connected events with a beginning, a series of possible steps or actions and an end point. They are generally derived from real world knowledge of the environment, the people involved -- their characteristics such as education, training, responsibilities, experiences, situaetc. --  and the kinds of actions they would engage with the systems and devices in development in order to accomplish a particular task. Scenarios also consider possible paths and actions that would lead to making a use error. Scenarios particularly worst-case scenarios can be an effective means for detecting possible use errors. With newly designed products and systems, this maybe one of the first means to identify possible use errors and determine their possible harm. Nevertheless, scenario-based use errors come from thought experiments and lack empirical validation.
• Brain-storming: Brain storming is an unstructured or free-form process of analysis to capture the possible use errors. Brain storming sessions can be a particularly useful means of capturing conditions and use errors that may not have been seen or consider using other methods. However use errors derived from brain storming are not based on empirical evidence. Nevertheless, those who uncover use errors by brain storming often have high levels of expertise and experience in the technical area under consideration.

Each process as its own value and every method to detect or originate possible use errors should be considered. And when reporting use errors in the submission, I suggest that where the reported use error originated should be included in the final submitted report. And a summary of where use errors originated should be included in the narrative.

Submission of the Human Engineering File to the FDA and Other Regulatory Bodies: Part II

These are the sections we'll cover in this article.

Sec	Contents
1	Conclusion The device has been found to be safe and effective for the intended users, uses and use environments. Brief summary of HFE/UE processes and results that support this conclusion Discussion of residual use-related risk
2	Descriptions of intended device users, uses, use environments, and training Intended user population(s) and meaningful differences in capabilities between multiple user populations that could affect user interactions with the device Intended use and operational contexts of use Use environments and conditions that could affect user interactions with the device     Training intended for users
3	Description of device user interface  Graphical representation of device and its user interface Description of device user interface Device labeling Overview of operational sequence of device and expected user interactions with user interface

With the exception of Section 1 each one of these sections will have a document or document attached to that section. Each section of the narrative document will be a summary of the foundational document or documents attached to that specific section. Make sure that the foundational documents are clearly referenced in each section.

Section Content Analysis

Section 1: Conclusion

This section is an abstract that includes the conclusion of the research program: that the device is "safe and effective" for those intended to use it. I suggest that this section include a brief description of the device at the beginning, its purpose and what medical issues it addresses. For example, if the medical device was a pacemaker, describe what it does, for whom it is intended and what medical issues it addresses. This description should be about one paragraph in length. And the paragraph should end with the "safe and effective" statement and that as to why this conclusion had been reached is briefly listed below in this section. You can add that a full and complete summary follows this section. 

A brief description of the device may well be helpful to the reviewer(s). Although submitted as a complete package that includes a full and detailed description of the device, the package is often split among several reviewers with specific specialties such as human factors. Therefore, the reviewer of the human engineering file may not read or even see the entire submission. Thus a brief description of the device may well provide some welcome context to the reviewer.

The brief summary of human engineering processes can be as simple as a list of steps that were performed. That could include observational studies/contextual analysis, structured interviews of members of the user population and/or subject-matter experts, prototype testing, use-error identification, etc.

Finally, this would be the place to disclose any lingering use-error problems that remain and how they're being addressed. Also, note that these lingering use-error problems have been determined to be low risk: before or after risk mitigations.

Section 2: Descriptions of intended device users, uses, use environments, and training

This section consists of a discussion who would be using the device -- their characteristics, training, education, certification, etc. -- where and how it would be used and the circumstances were it would be used. And finally, a description of the instructions or training that may be required in order to operate the device.

This section will likely be lengthy and comprehensive. Not as lengthy as the foundation document or documents, but it should include a significant portion of what that document(s) contains. You should include enough information to provide the reviewer with a full understanding of who is expected to use the device, how and when it should be used, where it's expected to be used and what positive outcomes would be expected when it is used. And finally what if any are the circumstances such as  the use environment that could how it might be used or effect whether or not it is used correctly.

Your foundational documents will likely include several specific device use scenarios. Include some of the most illustrative of those scenarios in your narrative. 

Devices are often targeted for use by specific classes of users in specific environments. For example, a device may be targeted for use by ICU nurses in the ICU. However, the device could conceivably be used in the general hospital setting by general practice nurses, especially inexperienced or recently graduated nurses. This section should discuss this possibility and what might be needed to be done to adapt the device to other environments and more groups of users with different skills and skill levels.

Finally, summarize the research evidence collected as well as the analysis performed that pertain to this section. Be sure to reference all the foundational documents as well as any internal or outside, published research not specifically performed for this device but applicable to this section. 

Section 3: Description of device user interface 

It may be worth considering lifting the entire description with the images from the foundation document and placing it in the narrative. And then consider what you can remove and still provide the reviewer with adequate information to evaluate the operation of the device. The entire user manual will likely not be necessary to be included in the narrative. You could consider having someone who is not familiar with the device read the description to determine if enough information has been provided. If the person can accurately describe how users would interact with the user interface in order to accomplish specific task (with the images of the user interface that you have provided), then your description is adequate.

Although not stated, this would be an acceptable section to summarize the design process, the evidence that lead to the finalized design and how that evidence was used to both create the initial design and iterate the design until it reached its final form. 

Submission of the Human Engineering File to the FDA and Other Regulatory Bodies

Background

About a year ago I was asked what I thought was the most difficult phase of the medical device human engineering process. Frankly, I'd never before considered such a question. I could not identify any phase of the process that I considered more difficult than any other. I considered whether early phase formative research and testing or risk identification, use errors and risk management would be the most difficult. No, actually for me these phases have always proven to be the most interesting phases in the process. Challenging, yes; difficult, no.

The questioner had an answer in mind: she believed that the validation testing was the most difficult. I disagreed. I believe that the validation testing was most often the least difficult of all the phases of the process. Why? Because validation comes at the end of the entire process and is based on all of the work that you've done previously to get one to the point of validation testing. Thus the procedure of a validation test should flow naturally and easily from the earlier work.

In the end, we agreed to disagree. However, the question never left me. 

I have finally come up with the answer. The most difficult phase of the process is the creation of the narrative for the regulatory reviewers. What I refer to is more than putting together the folder of documents of all of the human engineering related activities. It is the construction of a cohesive and understandable narrative the provides to a reviewer an overall view of the reasoning and logic of the steps taken and that the procedures performed that will demonstrate that the human engineering process was sound and resulting from it is a system that will be safe to use. This is the most difficult phase of the human engineering process, especially if writing the narrative comes at the end.

Human Engineering Pre-Market Submission

Here is the outline of what the FDA expects in a Human Engineering Pre-Market Submission as provided by the FDA on their website:

Sec.	Contents
1	Conclusion The device has been found to be safe and effective for the intended users, uses and use environments. Brief summary of HFE/UE processes and results that support this conclusion Discussion of residual use-related risk
2	Descriptions of intended device users, uses, use environments, and training Intended user population(s) and meaningful differences in capabilities between multiple user populations that could affect user interactions with the device Intended use and operational contexts of use Use environments and conditions that could affect user interactions with the device     Training intended for users
3	Description of device user interface  Graphical representation of device and its user interface Description of device user interface Device labeling Overview of operational sequence of device and expected user interactions with user interface
4	Summary of known use problems  Known use problems with previous models of the subject device Known use problems with similar devices, predicate devices or devices with similar user interface elements Design modifications implemented in response to  post-market use error problems
5	Analysis of hazards and risks associated with use of the device Potential use errors Potential harm and severity of harm that could result from each use error Risk management measures implemented to eliminate or reduce the risk Evidence of effectiveness of each risk management measure
6	Summary of preliminary analyses and evaluations Evaluation methods used Key results and design modifications implemented in response Key findings that informed the human factors validation test protocol
7	Description and categorization of critical tasks  Process used to identify critical tasks List and descriptions of critical tasks Categorization of critical tasks by severity of potential harm Descriptions of use scenarios that include critical tasks
8	Details of human factors validation testing Rationale for test type selected (i.e., simulated use, actual use or clinical study) Test environment and conditions of use Number and type of test participants Training provided to test participants and how it corresponded to real-world training levels Critical tasks and use scenarios included in testing Definition of successful performance of each test task  Description of data to be collected and methods for documenting observations and interview responses Test results: Observations of task performance and occurrences of use errors, close calls, and use problems  Test results: Feedback from interviews with test participants regarding device use, critical tasks, use errors, and problems (as applicable)   Description and analysis of all use errors and difficulties that could cause harm, root causes of the problems, and implications for additional risk elimination or reduction

Each one of the items listed will include one or more documents that address in some manner each one of the sub-points listed in each one of the items. You could place these documents into a folder, identify where they belong and submit this to the FDA or other regulatory body. However, I advocate something more: write an over arching narrative that takes the reader through the path of not only what was performed but why as well. The narrative can be and probably should be based on the outline above. (I'm sure there are exceptions, but beginning with the outline above should be your starting point.)

The narrative should provide a comprehensive flow that interconnects each of the phases and explains reasoning for what was done including the rationale behind the design and operation of the system as well as why it's appropriate for use by the identified user population as well as other likely populations who might encounter it.

Why Write a Narrative?

Is a narrative required as part of a submission? From all that I can tell: no, it's not a requirement as part of a submission to regulatory bodies. 

However, consider the fact that if you can't explain the logic of what you did to yourselves, how hard will it be for a reviewer to comprehend? And if the reviewer can't comprehend what you did -- including the reasoning and logic behind it -- could your submission be at risk for rejection? The answer is "yes," you may be putting your submission at risk for rejection.

The narrative is analogous to a completed jigsaw puzzle. A human engineering file without a narrative is analogous to just the jigsaw puzzle pieces. Yes everything is there, but what is it suppose to be? 

Submitting a human engineering file that includes a comprehensive narrative can insure that your submission is understandable: to you as well as your reviewers. It can insure that there are no gaps or issues that should have been included in your submission are left out. Again, going back to the jigsaw puzzle analogy, you don't know that you've got a missing piece or pieces until you've assembled the puzzle.

The narrative provides reviewers with framework to understand what you've done. Interestingly enough this will likely minimize any questions reviewer might have about your submission. And will likely minimize the likelihood that you'll get questions that you cannot answer.

One more thing to note: if your narrative is clear and comprehensive, it's likely that the reviewer or reviewers will often read no further or will simply scan the foundational documents to insure that the foundational documents do in fact support what is stated in the narrative. This could speed the regulatory review and acceptance. 

More Articles on This Topic

I'll be writing a series of articles on the topic of human engineering file submission narratives over the next week or two. I'll focus on specific areas of the narrative and discuss some of what I have done with regards to putting together narratives for submission to regulators. 

International Medical Device Database

For anyone interested in medical device safety, you should bookmark this website: https://medicaldevices.icij.org

It has been created by the International Consortium of Investigative Journalists to:

"Explore more than 70,000 Recalls, Safety Alerts and Field Safety Notices of medical devices and their connections with their manufacturers." 

Medical Design Article: FDA announces Medical Device Home use Initiative

As I was working on a human factors related article, this article from Medical Design appeared.  Here's the link to the article: http://medicaldesign.com/contract-manufacturing/fda-announces-medical-device-home-050310/

I thought that this article is interesting and telling with respect to how the FDA will assert it regulatory authority regarding usability issues. Here are a few quote from the article.

Recognizing that more patients of all ages are being discharged from hospitals to continue their medical treatment at home, the U.S. Food and Drug Administration announced an initiative to ensure that caregivers and patients safely use complex medical devices in the home. (My emphasis.) The initiative will develop guidance for manufacturers that intend to market such devices for home use, provide for post-market surveillance, and put in place other measures to encourage safe use of these products. The FDA is also developing education materials on home use of medical devices.

These home care patients often need medical devices and equipment such as hemodialysis equipment to treat kidney failure, wound therapy care, intravenous therapy devices, and ventilators.