Submission of the Human Engineering File to the FDA and Other Regulatory Bodies, Sections 6 and 7: Part V

I cover Sections 6 and 7 in this article as shown below:

6	Summary of preliminary analyses and evaluations Evaluation methods used Key results and design modifications implemented in response Key findings that informed the human factors validation test protocol
7	Description and categorization of critical tasks  Process used to identify critical tasks List and descriptions of critical tasks Categorization of critical tasks by severity of potential harm Descriptions of use scenarios that include critical tasks

I consider Sections 6 and 7 together because the information for these two sections should have come from the formative stage of the research and design process. These two sections could be combined into a single section. However, it is apparent that the FDA (and probably other regulatory bodies as well) considers Section 7, Description and categorization of critical tasks, as important enough to have its own, separate section. 

Importance of Getting It Right

The contents of these sections, the descriptions and explanations provided, can be the difference between: 

• An easy, unquestioned acceptance of what you've done or
• A difficult, question-riddled review of the work that you performed resulting in:
• Approval delays, 
• A reworking of the submitted materials 
• Requests for additional research to be performed, or 
• Rejection of the human engineering file 

To fully address what should be included in Sections 6 and 7, you need to examine your entire HE process in the context of the research and development program of your medical device or system and determine whether your HE process can adequately address reporting requirements of these two sections. These sections form the core of the report of your research and design process up to the point immediately before you begin your final phase of testing, namely verification and validation (summative) testing.

Section 6: Summary of the Preliminary Analysis and Evaluations

What Should be in Section 6

I briefly cover the points of what should be included in Section 6. Assuming that you are a human engineering professional, you should already have a reasonable understanding of the meaning of each of three requirements listed below. 

1. Evaluation methods used

This comprises the entire body of research performed including all of the data collected before the implementing a foundational or initial design, and all of the testing performed on the design.

2. Key results and design modifications implemented in response

What findings from your research lead to you to creating your initial design and what where the factors that lead you to modifying your design?

3. Key findings that informed the human factors validation test protocol

How did your arrive at creating your research protocol for summative/validation testing? How do you know that your validation protocol is appropriate and will verify that your system or device is safe for use?

That's the brief overview of what should be in Section 6. However, what should be included in Section 6 are the logical threads of justifications for doing what you did: for creating your research and development plan, the initial/foundational design and how you went about modifying that design. 
   
Don't be deceived by the seeming simplicity of Section 6. It is far more complicated and demands much more investigative and design process rigor than one might imagine. 

Human Engineering (HE): Research and Development

Section 6 is the section where you lay out all research and development performed in relationship to human engineering. Thus, Section 6 becomes the place where you make your case for the research that you performed and the design choices that you made. After reading Section 6, the reviewer should have a clear understanding and be in agreement with the research and design process that was undertaken. This includes the rationale for the research plan as proposed and undertaken including the rationale for any changes made to the plan on the basis of research findings. It will include the rationale for the design process, including the initial or foundational design and the reasons for changes made through the design iteration process.

Human factors is the study of how human interact with or operate systems and devices. Its fundamentally research. Human engineering incorporates the human factors, but encompasses and  incorporates design and the design process that should be at its foundation, driven by research. The research that directs and informs design and the design process includes field, laboratory, library, risk or research-based standards. And in the absence of the ability to collect empirical data: scenarios and interaction walk-throughs and analysis. 


You will need to defend your rationale for the specific research projects undertaken and the design choices made. Because the narrative is an overview, it's often a good place to explain the much of the logic for the research undertaken and the design choices made.

Defending HE Research and Design Planning and Choices

Adequate and effective justification of your research and development plan and design choices will often be the key to insuring unquestioned acceptance of your submission. Here are some suggestions:

1. Justifying the Research and Development Plan -- the means for creating a usable and low-likelihood use-error and low risk system or device. Reasoning and justifications for the creating a research and development plan for this system or device include:
• Compliance with IEC 62366 (part 1).
• Conformance to FDA HE program guidance (on the FDA website).
• Guidance from AAMI/ANSI HE-75
• Guidance from previous, similar and accepted plans 
• This system or device is a next generation release of a currently, commercially available product. Thus the research and development performed along with field collected data provide guidance for research and design plan for this next generation product.
2. Justification for performing specific research include:
• Planned research
• Research fits within the guidelines set within the research plan.
• Research is designed to answer specific research questions. Often during a research program, questions arise that may be human performance, design specific, etc. that may not have been specified in the research plan. Often times these types of studies are applicable to the research and development of a variety of device and systems. In this case the research is "question-driven." Those research questions need to be clearly defined out within the research protocol and become the clear justification for the research and the applicability and potential value of the findings.
• Findings from planned research suggest the need for new research not originally planned.
3. Justification for the foundational design: is initial design that is prototyped, usability tested and then iterated. The foundational design establishes the basic design philosophy (appearance and operation) that will likely be commercialized. While the foundational design will likely be updated and improved throughout the research and development process; fundamentally, it will likely maintain the same design philosophy. Thus, establishment of the foundational design maybe the most consequential step in the research and development process. Justifications for the foundational design include:
• Updated version of an earlier, accepted design: using the same design philosophy. Updates and improvement driven-by field research, customer feedback, research on the use of the system under actual conditions.
• Findings from formative research as defined by the research and development plan undertaken before initiating a design.
• Compliance with accepted design standards, e.g., AAMI HE75. (There are a wide array of design standards issued and accepted by the US agencies as well as other agencies of a variety of countries. When localization of a design is required, the design standards issued by the targeted country should be considered and referenced.)
4. Justification for changes made to the foundational and modified designs.
• Findings from prototype testing.
• Findings from expert reviewers: resulting from design walkthroughs/reviews and/or interactions with the device or system.
• Limited field tests of prototypes.
5. Justification that the design has reached the stage for verification and validation (summative) testing. And that a research protocol can be written that can effectively and realistically test the system or device to demonstrate that the system or device will be safe for use by members of the targeted population in the intended use environment(s).
• This is the hand-off point to the summative testing phase.
• Justification that that the system or device is read to hand off: The formative testing up to this point should have subjected the system or device to the all of the testing that it would be subjected-to multiple times. And the system or device should have passed those tests multiple times. Thus, if the research and development plan was properly executed nothing of any concern should come from verification and validation testing. If there are findings that are the least bit concerning, then it is time to reexamine your research and development planning and protocols. 
• Finally, if your formative testing, meaning all of the testing performed up to this point, has been comprehensive,  rigorous and complete, then that testing should dictate the verification and validation research protocols.

What Should be Included in the Section 6 Narrative

I suggest that your narrative should be written in the form of a story. It should be a narration that describes in a linear fashion (from the beginning to immediately before the validation step) what you did, why you did it: 

• if it's research, summarize what you did and what you found, 
• if it's your foundational design, provide an high level description of how you arrived at this design (include enough figures to be sure that a reviewer will understand your description) and
• if it's a design update, explain what change or changes were made and why.

Be sure to include references to your submitted materials in your HE file.

Section 7: Description and categorization of critical tasks

Identifying the critical tasks that will be performed on your system or device should be part of formative research. Often the ability to identify the set of critical tasks is beyond the expertise of the human engineering professional and identifying as well as categorizing the critical tasks requires the support of subject-matter experts (who should be included from the beginning of the formative research stage). My experience has been to integrate subject-matter experts into the research and design process from product inception.  

The list of requirements for Section 7 include:

1. Process used to identify critical tasks

With your subject-matter experts, describe the process used to identify your critical tasks. 

2. List and descriptions of critical tasks

Include with this your justifications and reasoning for this list. 

3. Categorization of critical tasks by severity of potential harm

In addition, if any of your critical tasks have the possibility of inflicting moderate to critical harm, I suggest that mitigations developed to minimize the likelihood that harm would ever occur. 


4. Descriptions of use scenarios that include critical tasks

These use scenarios should form a fundamental part of both your testing as well as justification and rationale for your design (and updates to your design.)

Section 7 Narrative

I suggest that in your narrative that you include a table with the information from items 2 and 3 above. I would add a brief summary of the process that was used to identify your critical tasks. Finally, include a reference to the use scenarios that include the critical tasks. You don't need to include them in your narrative, a reference should be sufficient. 

______________________
Note: I plan on periodically updating this article as I learn more and reconsider what I have written. With each update, I'll include at the top of this article, when it was updated and list some of the changes that I have made. 

Supplement to Part IV Submissions to FDA: Risk Management and Use Errors

Additional Thoughts:

As part of your risk analysis and use error identification, you should identify the circumstances or origin of the reported use error. 

• Origins of reports of use errors, in order of likely relevance:

1. Field reported use errors: These require special attention because they were discovered while in use by users under actual conditions. That's the reason why field reported use errors are the most important.They deserve special consideration, particularly if the use error was responsible for any harm and that the use error occurs significantly more often than originally predicted. Consider performing a root cause analysis to determine why the use error occurred. Most especially consider the use or environmental conditions and who made the use error to be factored into why the use error was made. Be sure to determine which assumptions regarding use, conditions for use and predicted user characteristics were violated. 
2. Errors reported from empirical studies: These are use errors that have been observed under laboratory or other kinds of testing conditions defined by researchers. Furthermore, these use errors are from members of the expected user population(s) who have the expected requisite level of education and training. Thus in testing sessions the conditions of use including the environment have been structured and manipulated by the researchers. The results of the research and the use errors detected may be valid, but narrow in scope in both situations of use, the use environment, actual user characteristics including education and training, etc. 
3. Analysis based:
• Scenarios: Scenarios are set of connected events with a beginning, a series of possible steps or actions and an end point. They are generally derived from real world knowledge of the environment, the people involved -- their characteristics such as education, training, responsibilities, experiences, situaetc. --  and the kinds of actions they would engage with the systems and devices in development in order to accomplish a particular task. Scenarios also consider possible paths and actions that would lead to making a use error. Scenarios particularly worst-case scenarios can be an effective means for detecting possible use errors. With newly designed products and systems, this maybe one of the first means to identify possible use errors and determine their possible harm. Nevertheless, scenario-based use errors come from thought experiments and lack empirical validation.
• Brain-storming: Brain storming is an unstructured or free-form process of analysis to capture the possible use errors. Brain storming sessions can be a particularly useful means of capturing conditions and use errors that may not have been seen or consider using other methods. However use errors derived from brain storming are not based on empirical evidence. Nevertheless, those who uncover use errors by brain storming often have high levels of expertise and experience in the technical area under consideration.

Each process as its own value and every method to detect or originate possible use errors should be considered. And when reporting use errors in the submission, I suggest that where the reported use error originated should be included in the final submitted report. And a summary of where use errors originated should be included in the narrative.

Submission of the Human Engineering File to the FDA and Other Regulatory Bodies: Part IV

Section 5: Analysis of hazards and risks associated with use of the device

This section is one of the most important sections of your submission. Your narrative should have both a summary of your findings and highlight any notable findings from your risk analysis and include any important steps taken to mitigate risks. 

5	Analysis of hazards and risks associated with use of the device Potential use errors Potential harm and severity of harm that could result from each use error Risk management measures implemented to eliminate or reduce the risk Evidence of effectiveness of each risk management measure

Risk assessment and management for human engineering for medical devices focuses on the identification and management of use errors. This is defined in IEC 62366 (part 1).  It is valuable for representatives from human engineering to be a part of the risk management team because of the additional insights that experienced human engineering professionals can bring to the process of identifying, managing and mitigating risks. However, the specific focus of the human engineering file with respect risk management is on use errors.

Narrative should be both a summary and a means to highlight events and areas of specific importance. This section can be relatively brief, but include the following information.  

Potential Use Errors, Identifying Harm and Severity of Harm

Identify the number of use errors discovered categorized according to their level of harm/severity. Any use errors categorized as high or critical should be highlighted. Be certain to include the method or methods used to identify and categorize potential use errors: this could include methods such as general system analysis, scenarios, field data, etc. 

Risk Management Measures Implemented to Eliminate or Reduce the Risk

Once you've identified the potential use errors, discuss what was done to mitigate or eliminate them. (Of Note: Root cause analysis is a particularly effective method for understanding use errors and correcting them.)  In the narrative you can make general statements regarding what was done to manage use errors of medium risk and lower. For High and Severe risk use errors, the narrative should include the specifics of how these risks were managed. Risk management can include anything from changes in design, system error trapping and error prevention measures to updates to instructions.

Evidence of Effectiveness of Each Risk Management Measure

Finally, you will need empirical data to demonstrate that the use errors identified have been properly addressed. That will require testing with members of the targeted user population. Your narrative should include a summary (abstract) of the study or studies that you performed. And be sure to provide clear references to the documentation included in your submission. 

Note: if your targeted user population is particularly narrow and tightly specified, consider including other members from a wider group. For example, if the targeted group consists of ICU nurses, consider including general hospital RNs in your user testing group. All too often, medical devices intended for one highly trained group often appear for use by others who may be technically astute but lack the specific training of the highly trained group.

A Word to the Wise ...

You should show that your human engineering program assessed risks, identified use errors and mitigated use errors long before reaching verification and validation testing. This should have been addressed in your formative research and by periodic testing before reaching the Verification and Validation stage of testing. V & V should not be the place to uncover use errors. It should be the place that validates all of the work you've done up to that point. Now, it's likely that V&V will uncover some areas of concern, but these should be relatively minor and addressable in a relatively easily. If you have discovered numerous problems at the point of V&V, then you have a problem with your human engineering process and revamping that process should become a major focus of your organization. 

Submission of the Human Engineering File to the FDA and Other Regulatory Bodies: Part III

In this article we cover section 4 of how to file the human engineering file to the FDA:

4	Summary of known use problems  Known use problems with previous models of the subject device Known use problems with similar devices, predicate devices or devices with similar user interface elements Design modifications implemented in response to  post-market use error problems

Section 4: Summary of known use problems

First, if this device is an update and replacement for an earlier version of the same device, summarize by category (if appropriate) the use related problems that have been detected. These will likely be problems reported from users in the field. This section of the narrative should reference field reports or summaries that should be included with this submission to your approving regulatory body. All problems need not be included in the summary, but be sure that your summary discloses the most concerning and potentially harmful problems that the new release of your device corrects -- and point this out. The narrative need not include specifics on how the use problems of the previous version have been corrected, but be sure to include references to the documentation included with the submission where this is described.

Second, the device being submitted for approval may well be one of many functionally identical or nearly identical devices. In the narrative identify any significant problems found in any of these devices. Be sure to mention how your device avoids or overcomes any of the significant known problems of other devices. This is particularly important if your device is your first version of this type of device. If your device is the first of its kind, then consider referencing devices with similar characteristics including ones your company has created. And address any field detected use related problems and how they have been addressed in the design of the device being submitted for approval.

Collecting information about competitors devices can be a long process. However, most companies I know have people who do competitive market analysis. I have found that much of the information related to other companies products including many issues and concerns has been collected and cataloged by competitive market analysts. Furthermore, information from competitive market analysts can often be an input to the user interface design process. Finally, if the device being submitted for approval has solved many if not most of the user interface issues found in other's devices, that's a competitive edge and something that marketing should want to know about and share with customers and potential customers.

The one issue of concern will be how much overlap should there be between the foundation documentation and the summary included in the narrative. Since the narrative is a summary and a guide for the reviewer, it may be acceptable to have this summary be found only in the narrative. Or the narrative could include a brief summary and a pointer to the foundation document or documents. There is no set rule in this matter. Whatever you chose, you will need to make clear to your reviewer the path you have chosen to take to make this information available. 

Submission of the Human Engineering File to the FDA and Other Regulatory Bodies: Part II

These are the sections we'll cover in this article.

Sec	Contents
1	Conclusion The device has been found to be safe and effective for the intended users, uses and use environments. Brief summary of HFE/UE processes and results that support this conclusion Discussion of residual use-related risk
2	Descriptions of intended device users, uses, use environments, and training Intended user population(s) and meaningful differences in capabilities between multiple user populations that could affect user interactions with the device Intended use and operational contexts of use Use environments and conditions that could affect user interactions with the device     Training intended for users
3	Description of device user interface  Graphical representation of device and its user interface Description of device user interface Device labeling Overview of operational sequence of device and expected user interactions with user interface

With the exception of Section 1 each one of these sections will have a document or document attached to that section. Each section of the narrative document will be a summary of the foundational document or documents attached to that specific section. Make sure that the foundational documents are clearly referenced in each section.

Section Content Analysis

Section 1: Conclusion

This section is an abstract that includes the conclusion of the research program: that the device is "safe and effective" for those intended to use it. I suggest that this section include a brief description of the device at the beginning, its purpose and what medical issues it addresses. For example, if the medical device was a pacemaker, describe what it does, for whom it is intended and what medical issues it addresses. This description should be about one paragraph in length. And the paragraph should end with the "safe and effective" statement and that as to why this conclusion had been reached is briefly listed below in this section. You can add that a full and complete summary follows this section. 

A brief description of the device may well be helpful to the reviewer(s). Although submitted as a complete package that includes a full and detailed description of the device, the package is often split among several reviewers with specific specialties such as human factors. Therefore, the reviewer of the human engineering file may not read or even see the entire submission. Thus a brief description of the device may well provide some welcome context to the reviewer.

The brief summary of human engineering processes can be as simple as a list of steps that were performed. That could include observational studies/contextual analysis, structured interviews of members of the user population and/or subject-matter experts, prototype testing, use-error identification, etc.

Finally, this would be the place to disclose any lingering use-error problems that remain and how they're being addressed. Also, note that these lingering use-error problems have been determined to be low risk: before or after risk mitigations.

Section 2: Descriptions of intended device users, uses, use environments, and training

This section consists of a discussion who would be using the device -- their characteristics, training, education, certification, etc. -- where and how it would be used and the circumstances were it would be used. And finally, a description of the instructions or training that may be required in order to operate the device.

This section will likely be lengthy and comprehensive. Not as lengthy as the foundation document or documents, but it should include a significant portion of what that document(s) contains. You should include enough information to provide the reviewer with a full understanding of who is expected to use the device, how and when it should be used, where it's expected to be used and what positive outcomes would be expected when it is used. And finally what if any are the circumstances such as  the use environment that could how it might be used or effect whether or not it is used correctly.

Your foundational documents will likely include several specific device use scenarios. Include some of the most illustrative of those scenarios in your narrative. 

Devices are often targeted for use by specific classes of users in specific environments. For example, a device may be targeted for use by ICU nurses in the ICU. However, the device could conceivably be used in the general hospital setting by general practice nurses, especially inexperienced or recently graduated nurses. This section should discuss this possibility and what might be needed to be done to adapt the device to other environments and more groups of users with different skills and skill levels.

Finally, summarize the research evidence collected as well as the analysis performed that pertain to this section. Be sure to reference all the foundational documents as well as any internal or outside, published research not specifically performed for this device but applicable to this section. 

Section 3: Description of device user interface 

It may be worth considering lifting the entire description with the images from the foundation document and placing it in the narrative. And then consider what you can remove and still provide the reviewer with adequate information to evaluate the operation of the device. The entire user manual will likely not be necessary to be included in the narrative. You could consider having someone who is not familiar with the device read the description to determine if enough information has been provided. If the person can accurately describe how users would interact with the user interface in order to accomplish specific task (with the images of the user interface that you have provided), then your description is adequate.

Although not stated, this would be an acceptable section to summarize the design process, the evidence that lead to the finalized design and how that evidence was used to both create the initial design and iterate the design until it reached its final form.