Submission of the Human Engineering File to the FDA and Other Regulatory Bodies, Section 8: Part VI

This is the easiest for me to cover largely because the requirements for the validation section are clearly spelled out in detail.

8

Details of human factors validation testing Rationale for test type selected (i.e., simulated use, actual use or clinical study) Test environment and conditions of use Number and type of test participants Training provided to test participants and how it corresponded to real-world training levels Critical tasks and use scenarios included in testing Definition of successful performance of each test task  Description of data to be collected and methods for documenting observations and interview responses Test results: Observations of task performance and occurrences of use errors, close calls, and use problems  Test results: Feedback from interviews with test participants regarding device use, critical tasks, use errors, and problems (as applicable)   Description and analysis of all use errors and difficulties that could cause harm, root causes of the problems, and implications for additional risk elimination or reduction

These requirements are largely self explanatory. However, I would like to make a few comments and additions.

• Validation testing -- including verification testing -- are often performed by outside consulting firms. Thus it is extremely important that you spell out how your testing should be performed and the measurements to be collected and reported. I've noted that often times consulting company is asked to write both the protocol and the testing script. This is a mistake. The organization that performed the work up to the validation testing stage should be responsible for creating the protocol and the script, because it is this organization that will be responsible for the submission of the HE file to the FDA and/or other regulatory bodies. It's important that the research and development as well as the submission be responsible and in full control of what takes place during the validation step.
• Verification and Validation testing. Verification testing takes place under laboratory conditions using as testing participants members of the targeted user population. This is an additional check on the usability of the system or device. Validation testing takes place in actual or simulated actual conditions -- with all the distractions and problems that users will likely encounter.
• Rationale for type of testing performed and the conditions chosen for validation testing can be extremely important especially if you have chosen a testing procedure less rigorous than performance testing under real or simulated real conditions. Consult IEC 62366 and AAMI HE75 for guidance.
• Testing procedure should insure that full testing of critical tasks are performed and likely to be repeatedly performed by testing participants.
• Suggested additional measurement: If your system or device has error trapping and redirecting capabilities, be sure to report how often these capabilities were triggered and if they enabled the testing participant to successfully complete the task. This could be labeled as: task successfully completed, close call. However, a system or device with the capability to protect against use errors is a capability worth pointing out. 

What to include in your narrative?

Include the abstract or abstracts of your validation testing in your narrative. 

If you haven't included any significant issues or root cause analysis in your abstract, be sure to include this in your narrative. Be sure you surface all issues or concerns in your narrative, if you don't it could appear to a reviewer that you're trying to hide any problems that you encountered. Even the appearance of hiding problems could cause problems with receiving approval for your system or device. 

Submission of the Human Engineering File to the FDA and Other Regulatory Bodies, Sections 6 and 7: Part V

I cover Sections 6 and 7 in this article as shown below:

6	Summary of preliminary analyses and evaluations Evaluation methods used Key results and design modifications implemented in response Key findings that informed the human factors validation test protocol
7	Description and categorization of critical tasks  Process used to identify critical tasks List and descriptions of critical tasks Categorization of critical tasks by severity of potential harm Descriptions of use scenarios that include critical tasks

I consider Sections 6 and 7 together because the information for these two sections should have come from the formative stage of the research and design process. These two sections could be combined into a single section. However, it is apparent that the FDA (and probably other regulatory bodies as well) considers Section 7, Description and categorization of critical tasks, as important enough to have its own, separate section. 

Importance of Getting It Right

The contents of these sections, the descriptions and explanations provided, can be the difference between: 

• An easy, unquestioned acceptance of what you've done or
• A difficult, question-riddled review of the work that you performed resulting in:
• Approval delays, 
• A reworking of the submitted materials 
• Requests for additional research to be performed, or 
• Rejection of the human engineering file 

To fully address what should be included in Sections 6 and 7, you need to examine your entire HE process in the context of the research and development program of your medical device or system and determine whether your HE process can adequately address reporting requirements of these two sections. These sections form the core of the report of your research and design process up to the point immediately before you begin your final phase of testing, namely verification and validation (summative) testing.

Section 6: Summary of the Preliminary Analysis and Evaluations

What Should be in Section 6

I briefly cover the points of what should be included in Section 6. Assuming that you are a human engineering professional, you should already have a reasonable understanding of the meaning of each of three requirements listed below. 

1. Evaluation methods used

This comprises the entire body of research performed including all of the data collected before the implementing a foundational or initial design, and all of the testing performed on the design.

2. Key results and design modifications implemented in response

What findings from your research lead to you to creating your initial design and what where the factors that lead you to modifying your design?

3. Key findings that informed the human factors validation test protocol

How did your arrive at creating your research protocol for summative/validation testing? How do you know that your validation protocol is appropriate and will verify that your system or device is safe for use?

That's the brief overview of what should be in Section 6. However, what should be included in Section 6 are the logical threads of justifications for doing what you did: for creating your research and development plan, the initial/foundational design and how you went about modifying that design. 
   
Don't be deceived by the seeming simplicity of Section 6. It is far more complicated and demands much more investigative and design process rigor than one might imagine. 

Human Engineering (HE): Research and Development

Section 6 is the section where you lay out all research and development performed in relationship to human engineering. Thus, Section 6 becomes the place where you make your case for the research that you performed and the design choices that you made. After reading Section 6, the reviewer should have a clear understanding and be in agreement with the research and design process that was undertaken. This includes the rationale for the research plan as proposed and undertaken including the rationale for any changes made to the plan on the basis of research findings. It will include the rationale for the design process, including the initial or foundational design and the reasons for changes made through the design iteration process.

Human factors is the study of how human interact with or operate systems and devices. Its fundamentally research. Human engineering incorporates the human factors, but encompasses and  incorporates design and the design process that should be at its foundation, driven by research. The research that directs and informs design and the design process includes field, laboratory, library, risk or research-based standards. And in the absence of the ability to collect empirical data: scenarios and interaction walk-throughs and analysis. 


You will need to defend your rationale for the specific research projects undertaken and the design choices made. Because the narrative is an overview, it's often a good place to explain the much of the logic for the research undertaken and the design choices made.

Defending HE Research and Design Planning and Choices

Adequate and effective justification of your research and development plan and design choices will often be the key to insuring unquestioned acceptance of your submission. Here are some suggestions:

1. Justifying the Research and Development Plan -- the means for creating a usable and low-likelihood use-error and low risk system or device. Reasoning and justifications for the creating a research and development plan for this system or device include:
• Compliance with IEC 62366 (part 1).
• Conformance to FDA HE program guidance (on the FDA website).
• Guidance from AAMI/ANSI HE-75
• Guidance from previous, similar and accepted plans 
• This system or device is a next generation release of a currently, commercially available product. Thus the research and development performed along with field collected data provide guidance for research and design plan for this next generation product.
2. Justification for performing specific research include:
• Planned research
• Research fits within the guidelines set within the research plan.
• Research is designed to answer specific research questions. Often during a research program, questions arise that may be human performance, design specific, etc. that may not have been specified in the research plan. Often times these types of studies are applicable to the research and development of a variety of device and systems. In this case the research is "question-driven." Those research questions need to be clearly defined out within the research protocol and become the clear justification for the research and the applicability and potential value of the findings.
• Findings from planned research suggest the need for new research not originally planned.
3. Justification for the foundational design: is initial design that is prototyped, usability tested and then iterated. The foundational design establishes the basic design philosophy (appearance and operation) that will likely be commercialized. While the foundational design will likely be updated and improved throughout the research and development process; fundamentally, it will likely maintain the same design philosophy. Thus, establishment of the foundational design maybe the most consequential step in the research and development process. Justifications for the foundational design include:
• Updated version of an earlier, accepted design: using the same design philosophy. Updates and improvement driven-by field research, customer feedback, research on the use of the system under actual conditions.
• Findings from formative research as defined by the research and development plan undertaken before initiating a design.
• Compliance with accepted design standards, e.g., AAMI HE75. (There are a wide array of design standards issued and accepted by the US agencies as well as other agencies of a variety of countries. When localization of a design is required, the design standards issued by the targeted country should be considered and referenced.)
4. Justification for changes made to the foundational and modified designs.
• Findings from prototype testing.
• Findings from expert reviewers: resulting from design walkthroughs/reviews and/or interactions with the device or system.
• Limited field tests of prototypes.
5. Justification that the design has reached the stage for verification and validation (summative) testing. And that a research protocol can be written that can effectively and realistically test the system or device to demonstrate that the system or device will be safe for use by members of the targeted population in the intended use environment(s).
• This is the hand-off point to the summative testing phase.
• Justification that that the system or device is read to hand off: The formative testing up to this point should have subjected the system or device to the all of the testing that it would be subjected-to multiple times. And the system or device should have passed those tests multiple times. Thus, if the research and development plan was properly executed nothing of any concern should come from verification and validation testing. If there are findings that are the least bit concerning, then it is time to reexamine your research and development planning and protocols. 
• Finally, if your formative testing, meaning all of the testing performed up to this point, has been comprehensive,  rigorous and complete, then that testing should dictate the verification and validation research protocols.

What Should be Included in the Section 6 Narrative

I suggest that your narrative should be written in the form of a story. It should be a narration that describes in a linear fashion (from the beginning to immediately before the validation step) what you did, why you did it: 

• if it's research, summarize what you did and what you found, 
• if it's your foundational design, provide an high level description of how you arrived at this design (include enough figures to be sure that a reviewer will understand your description) and
• if it's a design update, explain what change or changes were made and why.

Be sure to include references to your submitted materials in your HE file.

Section 7: Description and categorization of critical tasks

Identifying the critical tasks that will be performed on your system or device should be part of formative research. Often the ability to identify the set of critical tasks is beyond the expertise of the human engineering professional and identifying as well as categorizing the critical tasks requires the support of subject-matter experts (who should be included from the beginning of the formative research stage). My experience has been to integrate subject-matter experts into the research and design process from product inception.  

The list of requirements for Section 7 include:

1. Process used to identify critical tasks

With your subject-matter experts, describe the process used to identify your critical tasks. 

2. List and descriptions of critical tasks

Include with this your justifications and reasoning for this list. 

3. Categorization of critical tasks by severity of potential harm

In addition, if any of your critical tasks have the possibility of inflicting moderate to critical harm, I suggest that mitigations developed to minimize the likelihood that harm would ever occur. 


4. Descriptions of use scenarios that include critical tasks

These use scenarios should form a fundamental part of both your testing as well as justification and rationale for your design (and updates to your design.)

Section 7 Narrative

I suggest that in your narrative that you include a table with the information from items 2 and 3 above. I would add a brief summary of the process that was used to identify your critical tasks. Finally, include a reference to the use scenarios that include the critical tasks. You don't need to include them in your narrative, a reference should be sufficient. 

______________________
Note: I plan on periodically updating this article as I learn more and reconsider what I have written. With each update, I'll include at the top of this article, when it was updated and list some of the changes that I have made. 

Supplement to Part IV Submissions to FDA: Risk Management and Use Errors

Additional Thoughts:

As part of your risk analysis and use error identification, you should identify the circumstances or origin of the reported use error. 

• Origins of reports of use errors, in order of likely relevance:

1. Field reported use errors: These require special attention because they were discovered while in use by users under actual conditions. That's the reason why field reported use errors are the most important.They deserve special consideration, particularly if the use error was responsible for any harm and that the use error occurs significantly more often than originally predicted. Consider performing a root cause analysis to determine why the use error occurred. Most especially consider the use or environmental conditions and who made the use error to be factored into why the use error was made. Be sure to determine which assumptions regarding use, conditions for use and predicted user characteristics were violated. 
2. Errors reported from empirical studies: These are use errors that have been observed under laboratory or other kinds of testing conditions defined by researchers. Furthermore, these use errors are from members of the expected user population(s) who have the expected requisite level of education and training. Thus in testing sessions the conditions of use including the environment have been structured and manipulated by the researchers. The results of the research and the use errors detected may be valid, but narrow in scope in both situations of use, the use environment, actual user characteristics including education and training, etc. 
3. Analysis based:
• Scenarios: Scenarios are set of connected events with a beginning, a series of possible steps or actions and an end point. They are generally derived from real world knowledge of the environment, the people involved -- their characteristics such as education, training, responsibilities, experiences, situaetc. --  and the kinds of actions they would engage with the systems and devices in development in order to accomplish a particular task. Scenarios also consider possible paths and actions that would lead to making a use error. Scenarios particularly worst-case scenarios can be an effective means for detecting possible use errors. With newly designed products and systems, this maybe one of the first means to identify possible use errors and determine their possible harm. Nevertheless, scenario-based use errors come from thought experiments and lack empirical validation.
• Brain-storming: Brain storming is an unstructured or free-form process of analysis to capture the possible use errors. Brain storming sessions can be a particularly useful means of capturing conditions and use errors that may not have been seen or consider using other methods. However use errors derived from brain storming are not based on empirical evidence. Nevertheless, those who uncover use errors by brain storming often have high levels of expertise and experience in the technical area under consideration.

Each process as its own value and every method to detect or originate possible use errors should be considered. And when reporting use errors in the submission, I suggest that where the reported use error originated should be included in the final submitted report. And a summary of where use errors originated should be included in the narrative.

Submission of the Human Engineering File to the FDA and Other Regulatory Bodies: Part IV

Section 5: Analysis of hazards and risks associated with use of the device

This section is one of the most important sections of your submission. Your narrative should have both a summary of your findings and highlight any notable findings from your risk analysis and include any important steps taken to mitigate risks. 

5	Analysis of hazards and risks associated with use of the device Potential use errors Potential harm and severity of harm that could result from each use error Risk management measures implemented to eliminate or reduce the risk Evidence of effectiveness of each risk management measure

Risk assessment and management for human engineering for medical devices focuses on the identification and management of use errors. This is defined in IEC 62366 (part 1).  It is valuable for representatives from human engineering to be a part of the risk management team because of the additional insights that experienced human engineering professionals can bring to the process of identifying, managing and mitigating risks. However, the specific focus of the human engineering file with respect risk management is on use errors.

Narrative should be both a summary and a means to highlight events and areas of specific importance. This section can be relatively brief, but include the following information.  

Potential Use Errors, Identifying Harm and Severity of Harm

Identify the number of use errors discovered categorized according to their level of harm/severity. Any use errors categorized as high or critical should be highlighted. Be certain to include the method or methods used to identify and categorize potential use errors: this could include methods such as general system analysis, scenarios, field data, etc. 

Risk Management Measures Implemented to Eliminate or Reduce the Risk

Once you've identified the potential use errors, discuss what was done to mitigate or eliminate them. (Of Note: Root cause analysis is a particularly effective method for understanding use errors and correcting them.)  In the narrative you can make general statements regarding what was done to manage use errors of medium risk and lower. For High and Severe risk use errors, the narrative should include the specifics of how these risks were managed. Risk management can include anything from changes in design, system error trapping and error prevention measures to updates to instructions.

Evidence of Effectiveness of Each Risk Management Measure

Finally, you will need empirical data to demonstrate that the use errors identified have been properly addressed. That will require testing with members of the targeted user population. Your narrative should include a summary (abstract) of the study or studies that you performed. And be sure to provide clear references to the documentation included in your submission. 

Note: if your targeted user population is particularly narrow and tightly specified, consider including other members from a wider group. For example, if the targeted group consists of ICU nurses, consider including general hospital RNs in your user testing group. All too often, medical devices intended for one highly trained group often appear for use by others who may be technically astute but lack the specific training of the highly trained group.

A Word to the Wise ...

You should show that your human engineering program assessed risks, identified use errors and mitigated use errors long before reaching verification and validation testing. This should have been addressed in your formative research and by periodic testing before reaching the Verification and Validation stage of testing. V & V should not be the place to uncover use errors. It should be the place that validates all of the work you've done up to that point. Now, it's likely that V&V will uncover some areas of concern, but these should be relatively minor and addressable in a relatively easily. If you have discovered numerous problems at the point of V&V, then you have a problem with your human engineering process and revamping that process should become a major focus of your organization. 

Apple Watch 5: Heart Monitoring Capabilities -- Afib

The Apple Watch 5 has a heart rhythm monitoring capability that is tuned to detecting the presence of atrial fibrillation, AKA, Afib. Apple categorically states that the watch is unable to detect a heart attack. (And by implication, the likelihood of a heart attack occurring within minutes or hours.)

You have to manually enable your heart monitoring system (Watch and iPhone) to detect Afib. This not part of the default configuration. Here's the link for setting it up: https://support.apple.com/en-us/HT208931#afib

Here's what Apple says about the capabilities of their system and note that it requires both the Apple Watch 5 and an iPhone: 

INDICATIONS FOR USE (NON-EU REGIONS)

The Irregular Rhythm Notification Feature is a software-only mobile medical application that is intended to be used with the Apple Watch. The feature analyzes pulse rate data to identify episodes of irregular heart rhythms suggestive of atrial fibrillation (AF) and provides a notification to the user. The feature is intended for over-the-counter (OTC) use. It is not intended to provide a notification on every episode of irregular rhythm suggestive of AF and the absence of a notification is not intended to indicate no disease process is present; rather the feature is intended to opportunistically surface a notification of possible AF when sufficient data are available for analysis. These data are only captured when the user is still. Along with the user’s risk factors, the feature can be used to supplement the decision for AF screening. The feature is not intended to replace traditional methods of diagnosis or treatment.

The feature has not been tested for and is not intended for use in people under 22 years of age. It is also not intended for use in individuals previously diagnosed with AF.

INTENDED PURPOSE (EU REGION)

Intended Use

The Irregular Rhythm Notification Feature (IRNF) is intended to pre-screen and notify the user of the presence of irregular rhythms suggestive of atrial fibrillation (AF). The feature can be used to supplement a clinician’s decision to screen for possible AF. The feature is intended for over-the-counter (OTC) use.

The feature has not been tested for and is not intended for use in people under 22 years of age. It is also not intended for use in individuals previously diagnosed with AF.

Indications

The feature is indicated to pre-screen for irregular rhythms suggestive of AF for anyone aged 22 years and over.

USING THE IRREGULAR RHYTHM NOTIFICATION FEATURE Set-Up/On-boarding

• Open the Health app on your iPhone.
• Navigate to “Heart”, then select “Irregular Rhythm Notifications”.
• Follow the onscreen instructions.

Receiving a Notification

Once the feature is turned on, you will receive a notification if the feature identified a heart rhythm suggestive of AF and confirmed it on multiple readings.
If you have not been diagnosed with AF by a GP, you should discuss the notification with your doctor.

All data collected and analysed by the Irregular Rhythm Notification Feature is saved to the Health app on your iPhone. If you choose to, you can share that information by exporting your health data in the Health app.

SAFETY AND PERFORMANCE

In a study of 226 participants aged 22 years or older who had received an AF notification while wearing Apple Watch and subsequently wore an electrocardiogram (ECG) patch for approximately one week, 41.6% (94/226) had AF detected by ECG patch. During concurrent wear of Apple Watch and an ECG patch, 57/226 participants received an AF notification. Of those, 78.9% (45/57) showed concordant AF on the ECG patch and 98.2% (56/57) showed AF and other clinically relevant arrhythmias. A total of 370 irregular rhythm notifications with readable ECG patch data was received by the 57 participants. Of those 370 notifications, 322 (87.0%) were assessed to be AF, 47 (12.7%) were arrhythmias other than AF and 1 (0.3%) was sinus rhythm. These results demonstrate that, while in the majority of cases the notification will accurately represent the presence of AF, in some instances, a notification may indicate the presence of an arrhythmia other than AF. No serious device adverse effects were observed.

CAUTIONS

The Irregular Rhythm Notification Feature cannot detect heart attacks. If you ever experience chest pain, pressure, tightness or what you think is a heart attack, call emergency services.

The Irregular Rhythm Notification Feature is not constantly looking for AF and should not be relied on as a continuous monitor. This means the feature cannot detect all instances of AF and people with AF may not get a notification.

• Not intended for use by individuals previously diagnosed with AF.
• Notifications made by this feature are potential findings, not a complete diagnosis of cardiac conditions. All notifications should be reviewed by a medical professional for clinical decision making.
• Apple does not guarantee that you are not experiencing an arrhythmia or other health conditions even in the absence of an irregular rhythm notification. You should notify your GP if you experience any changes to your health.
• For best results, make sure your Apple Watch fits snugly on top of your wrist. The heart rate sensor should stay close to your skin.

From the information provided I am unable to determine how the Afib monitoring system detects Afib. It does seem use an additional capability beyond heart rate system, but from what little I can understand, it uses software running on either the watch and/or the iPhone and uses as input the data from the heart rate system.

I have no idea what algorithms the Apple heart monitoring system is using to detect atrial fibrillation (AF), but if you read the study above, you'll note that apparently, the Apple system has significant false positive rate. Walking through the study, to qualify as a subject for the study, you had to have had a positive indication of AF by the Apple system. That's the one clear message from the study. Another clear message is that both the Apple system and the AF patch can detect heart arrhythmia  other than AF, but what those were is unclear. Unfortunately the way the data is reported does not provide full clarity into the procedure and results. So there's not much more that I can comfortably conclude.

I feel comfortable stating that if you're wearing the Apple Watch and using the AF detection system and you get an AF indication, it's worth your time to get it checked out even knowing full well that the indication is more than likely to be a false positive.

However, high AF false positive rate of nearly 60% is concerning from the standpoint of those who have the Apple AF detection system activated and receive false positive indications. Information like this gets around and users may have tendency to ignore the AF indications when in fact they should be paying attention to them. To curb the possibility that someone ignores an accurately reported AF indication from the Apple system, it would behove Apple to include with the AF notification a check list displayed on the iPhone the walk the user through to determine if in fact this is an AF event.

Submission of the Human Engineering File to the FDA and Other Regulatory Bodies: Part III

In this article we cover section 4 of how to file the human engineering file to the FDA:

4	Summary of known use problems  Known use problems with previous models of the subject device Known use problems with similar devices, predicate devices or devices with similar user interface elements Design modifications implemented in response to  post-market use error problems

Section 4: Summary of known use problems

First, if this device is an update and replacement for an earlier version of the same device, summarize by category (if appropriate) the use related problems that have been detected. These will likely be problems reported from users in the field. This section of the narrative should reference field reports or summaries that should be included with this submission to your approving regulatory body. All problems need not be included in the summary, but be sure that your summary discloses the most concerning and potentially harmful problems that the new release of your device corrects -- and point this out. The narrative need not include specifics on how the use problems of the previous version have been corrected, but be sure to include references to the documentation included with the submission where this is described.

Second, the device being submitted for approval may well be one of many functionally identical or nearly identical devices. In the narrative identify any significant problems found in any of these devices. Be sure to mention how your device avoids or overcomes any of the significant known problems of other devices. This is particularly important if your device is your first version of this type of device. If your device is the first of its kind, then consider referencing devices with similar characteristics including ones your company has created. And address any field detected use related problems and how they have been addressed in the design of the device being submitted for approval.

Collecting information about competitors devices can be a long process. However, most companies I know have people who do competitive market analysis. I have found that much of the information related to other companies products including many issues and concerns has been collected and cataloged by competitive market analysts. Furthermore, information from competitive market analysts can often be an input to the user interface design process. Finally, if the device being submitted for approval has solved many if not most of the user interface issues found in other's devices, that's a competitive edge and something that marketing should want to know about and share with customers and potential customers.

The one issue of concern will be how much overlap should there be between the foundation documentation and the summary included in the narrative. Since the narrative is a summary and a guide for the reviewer, it may be acceptable to have this summary be found only in the narrative. Or the narrative could include a brief summary and a pointer to the foundation document or documents. There is no set rule in this matter. Whatever you chose, you will need to make clear to your reviewer the path you have chosen to take to make this information available. 

Submission of the Human Engineering File to the FDA and Other Regulatory Bodies: Part II

These are the sections we'll cover in this article.

Sec	Contents
1	Conclusion The device has been found to be safe and effective for the intended users, uses and use environments. Brief summary of HFE/UE processes and results that support this conclusion Discussion of residual use-related risk
2	Descriptions of intended device users, uses, use environments, and training Intended user population(s) and meaningful differences in capabilities between multiple user populations that could affect user interactions with the device Intended use and operational contexts of use Use environments and conditions that could affect user interactions with the device     Training intended for users
3	Description of device user interface  Graphical representation of device and its user interface Description of device user interface Device labeling Overview of operational sequence of device and expected user interactions with user interface

With the exception of Section 1 each one of these sections will have a document or document attached to that section. Each section of the narrative document will be a summary of the foundational document or documents attached to that specific section. Make sure that the foundational documents are clearly referenced in each section.

Section Content Analysis

Section 1: Conclusion

This section is an abstract that includes the conclusion of the research program: that the device is "safe and effective" for those intended to use it. I suggest that this section include a brief description of the device at the beginning, its purpose and what medical issues it addresses. For example, if the medical device was a pacemaker, describe what it does, for whom it is intended and what medical issues it addresses. This description should be about one paragraph in length. And the paragraph should end with the "safe and effective" statement and that as to why this conclusion had been reached is briefly listed below in this section. You can add that a full and complete summary follows this section. 

A brief description of the device may well be helpful to the reviewer(s). Although submitted as a complete package that includes a full and detailed description of the device, the package is often split among several reviewers with specific specialties such as human factors. Therefore, the reviewer of the human engineering file may not read or even see the entire submission. Thus a brief description of the device may well provide some welcome context to the reviewer.

The brief summary of human engineering processes can be as simple as a list of steps that were performed. That could include observational studies/contextual analysis, structured interviews of members of the user population and/or subject-matter experts, prototype testing, use-error identification, etc.

Finally, this would be the place to disclose any lingering use-error problems that remain and how they're being addressed. Also, note that these lingering use-error problems have been determined to be low risk: before or after risk mitigations.

Section 2: Descriptions of intended device users, uses, use environments, and training

This section consists of a discussion who would be using the device -- their characteristics, training, education, certification, etc. -- where and how it would be used and the circumstances were it would be used. And finally, a description of the instructions or training that may be required in order to operate the device.

This section will likely be lengthy and comprehensive. Not as lengthy as the foundation document or documents, but it should include a significant portion of what that document(s) contains. You should include enough information to provide the reviewer with a full understanding of who is expected to use the device, how and when it should be used, where it's expected to be used and what positive outcomes would be expected when it is used. And finally what if any are the circumstances such as  the use environment that could how it might be used or effect whether or not it is used correctly.

Your foundational documents will likely include several specific device use scenarios. Include some of the most illustrative of those scenarios in your narrative. 

Devices are often targeted for use by specific classes of users in specific environments. For example, a device may be targeted for use by ICU nurses in the ICU. However, the device could conceivably be used in the general hospital setting by general practice nurses, especially inexperienced or recently graduated nurses. This section should discuss this possibility and what might be needed to be done to adapt the device to other environments and more groups of users with different skills and skill levels.

Finally, summarize the research evidence collected as well as the analysis performed that pertain to this section. Be sure to reference all the foundational documents as well as any internal or outside, published research not specifically performed for this device but applicable to this section. 

Section 3: Description of device user interface 

It may be worth considering lifting the entire description with the images from the foundation document and placing it in the narrative. And then consider what you can remove and still provide the reviewer with adequate information to evaluate the operation of the device. The entire user manual will likely not be necessary to be included in the narrative. You could consider having someone who is not familiar with the device read the description to determine if enough information has been provided. If the person can accurately describe how users would interact with the user interface in order to accomplish specific task (with the images of the user interface that you have provided), then your description is adequate.

Although not stated, this would be an acceptable section to summarize the design process, the evidence that lead to the finalized design and how that evidence was used to both create the initial design and iterate the design until it reached its final form. 

Article: Year of Telehealth

Here's an article telling us what lots of us already have learned, that telehealth is an up a coming method of providing effective and cost-effective as well as continuous medical care where ever a patient may be. Here's the link to the article: https://www.beckershospitalreview.com/telehealth/dr-toby-cosgrove-2019-will-be-the-year-of-telehealth.html

Here's a quote from the article that I think is of interest:

"[Oakland, Calif.-based Kaiser Permanente] is seeing over 50 percent of their patients distantly," Dr. Cosgrove told CNBC.

What Cosgrove isn't telling us is how telehealth is being provided. Telehealth is pretty loosely defined. It can mean that patients have access to a health care provider through chat or the telephone. Or it can mean something more sophisticated such as continuous medical-device communication and automated monitoring. One way or another telehealth is clearly on the rise and will likely become the standard for providing care.

Apple Watch 4: New York Times Review

Here's an article reviewing the Apple Watch 4 that was published in the New York Times on 19 September 2018.

Here's the link: https://www.nytimes.com/2018/09/19/technology/personaltech/apple-watch-series-4-review-health.html?em_pos=large&emc=edit_ct_20180920&nl=technology&nlid=67594383edit_ct_20180920&ref=headline&te=1

As of the posting of this article, Apple has yet to release their ECG app -- the thing I guess that most of us have interest with regards to the Apple Watch 4. It's the one thing that moves the Apple Watch 4 from a consumer to a medical device and a hardware platform on which to base medical applications and services. So, until the ECG application is available, I'm holding off on reviewing the Apple Watch 4.

Apple Watch 4 -- FDA Announcement: Statement from FDA Commissioner Scott Gottlieb, M.D., and Center for Devices and Radiological Health Director Jeff Shuren, M.D., J.D., on agency efforts to work with tech industry to spur innovation in digital health

The FDA just provided what amounts to a "shout-out" to companies that design and manufacture intelligent, wearable devices that include medically-related monitoring devices and specifically, the Apple Watch 4.

Here's the link to the FDA statement: https://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm620246.htm

And here's an interesting quote from the announcement:

... [There have come] a new swath of companies that are investing in these new opportunities [e.g., wearable, intelligent monitoring devices measuring medically-related, physiological characteristics with analysis capabilities.] These firms may be new to health care products and may not be accustomed to navigating the regulatory landscape that has traditionally surrounded these areas. A great example is the announcement of two mobile medical apps designed by Apple to work on the Apple Watch. One app creates an electrocardiogram, similar to traditional electrocardiograms, to detect the presence of atrial fibrillation and regular heart rhythm, while the other app analyzes pulse rate data to identify irregular heart rhythms suggestive of atrial fibrillation and notify the user. The FDA worked closely with the company as they developed and tested these software products, which may help millions of users identify health concerns more quickly. Health care products on ubiquitous devices, like smart watches, may help users seek treatment earlier and will truly empower them with more information about their health.

---------------
I find it interesting that Dr. Gottlieb states that the Apple Watch analyzes pulse rate data, not the ECG, to detect "rhythms suggestive of atrial fibrillation." Yeah, that's a way to do it, but analysis of the ECG is a much better way. When I do a deep dive on the Apple Watch 4, I'll look into this and questions like it.

Apple Watch 4, Preview of Medical-Monitoring Features

Here's an article regarding the Apple Watch 4 and what are suppose to be built in medical monitoring features.

Here's the link: https://www.mobihealthnews.com/content/apple-watch-series-4-will-have-fda-cleared-ecg-fall-detection?mkt_tok=eyJpIjoiTkRVMk0yVmxNamsyWkRneiIsInQiOiJjWXRoaVpENmhJYlBRNFlzVVBYZ3hrc0VEVFdsYmNLUG1FQUIrQmcyMnVHMTRwSnBORDh6cW1Da1kzbjJqS2JxbHcydjRuTk0zaG5qRzBvMFR1MmdiMmZyNGhyXC9SZmYyYkduaSs5R0tyRG85TXkrMHVxTnFFYXFrVE5jWHpIRWwifQ%3D%3D

Here's the list of new medically-related features:

1. ECG (30 second rhythm "strip")
2. A-Fib detection (of course, if you're paying attention and you know the symptoms, you'll probably know sooner than the watch.)
3. Fall detection (as in when the person falls, the watch detects that it has occurred)

All information is sent back to Apple Health Records where all this information be accessible to a physician/cardiologist.

Apple has received FDA approval, according to the article. 

I'm not going to comment until I've had a little more time to study the Apple Watch 4 except to say, if you can detect A-Fib, then why not V-Fib? V-Fib is much more life threatening. Also too, if you've got a 30 second rhythm snap shot, you can do a lot with that. 

I'll touch on these and other questions regarding the Apple Watch 4 and Apple's effort to product a remote medical monitoring device and medical monitoring system later. 

UK Perspective Regarding FDA Regulatory Requirements

A Linked-In colleague posted a link to this article. I read it and found it interesting enough to post the link and comment on it. It's by a UK publication and discusses the FDA regulatory process as it relates to Human Engineering requirements for device approval for commercialization.

Here's the link:
http://www.emdt.co.uk/daily-buzz/what-are-fda-usability-testing-requirements-device-approval?cid=nl.qmed01.20150325

In addition, I provide my own perspective on the article in the "Commentary" section below. I do not critique the article. I only attempt to expand on a few points from it.

But first, a brief summary of the article.

Article Summary

Medical errors have become an increasing concern of the FDA. I became interested in medical errors when I was a consultant at St. Jude Medical Cardiac Rhythm Division in Sylmar, CA. During my time at St. Jude (2009-2010), deaths by medical error were being reported as being 100,000 to 120,000 per year. Last year, I posted links to two articles that stated that deaths by medical errors could be closer to 400,000 per year. (http://medicalremoteprogramming.blogspot.com/2014/07/rip-death-by-medical-error-400000-year.html)

It has been noted by the FDA a large proportion of medical errors can be attributed to poorly designed medical device user interfaces. Since a fundamental mission of the FDA is increasing patient safety and reducing injuries and fatalities in the practice of medicine, the FDA has begun placing greater emphasis on improving the usability of medical device user interfaces.

This article provides measures that show the FDA's increasing emphasis on usability and human factors issues by showing the increasing frequency that companies seeking medical device clearance for the US market mention the terms "usability" and "human factors." Figure 1 from the article clearly shows the increasing usage of these terms in company filings.

The focus should be on the trends, not the absolute numbers because not all filing documents have been included in the count. But the trend clearly shows an increased emphasis by companies to increasingly use the terms "usability" and "human factors" in their filings with the FDA. The two figures that follow suggest the degree that companies have incorporated the FDA prescribed human factors engineering process and design guidance documentation.

The documents listed below are specifically targeted to defining and supporting the human factors engineering process and the development of the Human Engineer File that's included as part of a company's filing to the FDA.

• ISO 62366, Medical Devices - Application of Usability Engineering to Medical Devices
• AAMI / ANSI HE75:2009, Human Factors Engineering - Design of Medical Devices (General)

I'll discuss the documents above in greater detail and describe how they're intended to fit within the human factors engineering process when developing medical devices.

• IEC 60601-1-6 Medical electrical equipment - Part 1-6 General requirements for Safety - Collateral standard: Usability
• IEC 60601-1-8 Ed. 1, Medical Electrical Equipment - Part 1-8: General Requirements for Safety - Collateral Standard: Alarm Systems - Requirements, Tests and Guidance - General Requirements and Guidelines for Alarm Systems in Medical Equipment (General)

The two documents above are engineering standards. They're engineering specifications that medical devices must meet. They are technical and specific.

I show Figure 3 from the article before showing Figure 2. 

The increasing reference of 60601-1-8 is not surprising given the increased emphasis on safety. My real interest is in the significant increase in reference to ISO 62366. As mentioned, this is process standard the lays out how human factors engineering should be engaged to reduce "use errors." The emphasis in this standard is on the reduction of risk. Risk management is extremely well embedded in the medical device design and engineering process. It would seem that from a cultural perspective, ISO 62366 fits with the medical device engineering process. 

I want to contrast the dramatic, increasing references to ISO 62366 with the references to AAMI/ANSI HE75 shown in Figure 2 below.

References to AAMI/ANSI HE75 rise and fall from 2010 to 2013 instead of a steady upward trend that you see with ISO 62366 in Figure 3. I would like to emphasize that ISO 62366 and AAMI/ANSI HE75 should be considered as companion documents. (I'll expand on this in the Commentary section below.)

Commentary

The article does support the contention that the FDA and the companies it regulates are paying increasing attention to usability and human factors. That they're paying enough attention is another matter entirely. As new medical devices are introduced we should see two things. First, the use error rate for the newly introduced medical devices (once users have adapted to them) should decline in relationship to other similar devices currently in use. Second, we should see over time the number of per year of deaths and injuries from medical errors begin to decline. This will take time to detect.

Without a doubt, the push by the FDA to define a human engineering process in the design and testing of medical devices, and to press for testing under actual or simulated conditions is needed. In many ways the FDA is mirroring many of the processes that have already been adopted by the US Department of Defense (DoD) in the area of human engineering. Admittedly, the DoD doesn't always get it right, there is an understanding within the DoD that it is important ... life saving, battle-winning important ... to insure that those at the controls can do their jobs quickly, effectively and with as few errors as possible.  So from that standpoint, the FDA has adopted processes from programs that have proven effective. But the FDA has just passed the starting line. And much more will be required going forward.

ISO 62366 vs AAMI/ANSI HE75

As I mentioned earlier ISO 62366 and AAMI/ANSI HE75 should be consider complementary or companion documents. HE75 is a much larger document than 62366 and includes a significant amount of device design guidance and guidelines. 62366 is almost entirely a process document that's devoted to directing how to go about managing the research and development process of a medical device. In addition, the focus of 62366 is managing risks, risks in the realm of reducing use errors.

I found it interesting that references to HE75 were not increasing at the rate as references to 62366. I would have expected Figures 2 and 3 to have a similar appearance with respect to 62366 and HE75 in large part because the documents significantly overlap. In fact I might have reasonably expected references to HE75 to outpace 62366 because HE75 includes design specific guidelines in addition.

One possible reason for references to HE75 not being referenced in the same accelerated way as HE75 may have to do with the fact that the European Union has not adopted HE75, so it's required for medical devices that will be marketed in the EU (CE).  (I am currently unaware of the regulatory requirements of other countries on this matter.) Medical device companies are international companies and the documents that they file in one country are generally the same in each country. Thus since the EU hasn't adopted HE75, references to HE75 and HE75's use as a foundational process and design document may be less.

DESIGN RATIONALE

I'm not sure that this is true at this point in time, but I am certain that the following will be true going forward at some time in the future. I believe that the FDA will hold companies to account for their user interface designs. I believe that the FDA will demand that companies clearly define how they came up with their user interface designs and that those designs are well-grounded in empirical evidence.

This is what I mean ... the FDA will demand that the design choices ... these include: controls, placement of controls, number of controls, actions performed by controls, the way the control responds, methods for interacting with the device (e. g., touch screen, buttons, mouse), size of the display, etc. ... for medical device user interfaces must be grounded in empirical data.

Commercial websites are often designed by graphic artists. Often times the design of webpages reflect the artist's aesthetic sensibilities. Layout appear they way that they do because they look good.

I believe that the FDA will require that user interface designs for medical devices have an empirically grounded design rationale. Companies will be required to point to specific research finding to justify the design and the design choices that they made. Furthermore, as the design of the user interface evolves with each iteration of testing, the FDA will require that changes to the design be based on research findings.

Finally, I believe that soon if it is not occurring already, that the FDA will require:

1. That companies submit documentation to show in detail the full evolutionary design process beginning from product inception, including ...
2. Detailed pre-design research ... population(s), method(s), research questions and rationale, etc ... as well as the findings and what they suggest for the design of the user interface
3. A design that includes with a full discussion of the design rationale ... why was it designed the way it was ... 
4. A detailed description of the evolution of the design that include full and clear justification(s) for each change in the design ... and require that changes be grounded empirical data 
5. A full description of pre-commercialization testing process and method ... with a clear justification for why this testing meets FDA testing requirements
6. And a complete and clear analysis of the testing data.

What I'm suggesting above is that the process of designing and testing a medical device user interface should be more than going through the prescribed steps, collecting the data, doing the tests, etc. There should be a clear thread that ties all the steps together. When in a subsequent step, one should be able to point back to the previous steps for the rationale to explain why the user interface was designed to appear and operate the way it does ... to this point.

As near as I can tell, what I described above is rigorous than is currently required by the FDA. However, I believe that it would be in any company's best interest to follow what I've suggested because there may come a time when the FDA's enforcement becomes more rigorous. 

Another reason may be lawsuits. If a company can show that it went beyond the FDA's regulatory requirements at the time, those suing would likely have less of a chance of collecting damages. And if damages were awarded, they may likely be lower. Also, if the company went beyond the FDA requirements, it would be likely that there would be fewer people injured and that should lower damages.

FINALLY

This article has been a springboard for me to discuss a number of topics related to human engineering for medical devices user interfaces. This topic will remain a central part of this blog. I'll return this within a week or two, and discuss in depth other topics related to the human engineering process for medical device user interfaces. 

How This Blog Got Going: MRI Safe and Conditional Pacemakers, Reprise

I have decided to return to the thing that I was working on when I started this blog ... an MRI conditional pacemaker. Specifically, an MRI conditional pacemaker for St. Jude Medical. At the time I was Lead Human Engineering Clinical Systems Engineer on this project. Before I go any further I would like to distinguish between MRI conditional and MRI safe devices. It is important to distinguish between the two.

MRI Conditional v. MRI Safe

Having an MRI safe implanted cardiac device is the ideal situation. If the cardiac device is MRI safe, it means that a device patient can be "popped" into an MRI without any changes to the device. For the patient it's just like the person does not have an implanted device. The only difference is that the resulting imagery from the MRI around the device may not be as good if the person did not have an implanted device. 

An MRI conditional device presents some significant procedural challenges to all those involved. If a person has an MRI conditional device, certain conditions must be met before the device patient is allowed to enter the MRI. When I was working at St. Jude Medical, changes in the settings that operate the device are required before the patient enters the MRI. Once scanning is complete, the settings need to be changed back to their normal, operational settings.

As of publication of this article, only one medical device company has a commercially available MRI safe pacemaker, Biotronik. St. Jude Medical and Medtronic have commercially available MRI conditional devices. 

When I was work at St. Jude, the only cardiac device being engineered to permit patients to have MRI scans were pacemakers. At the time ICDs and CRTs were not considered for MRI compatibility. However, apparently, Biotronik has developed an MRI conditional ICD that is commercially available ... at least in Europe.

There are other issues regarding MRI compatibility such as whether there are limits on the area that can be scanned a cardiac device patient ... something other than a full body scan. The allowable limits on how much can be scanned are continually in flux. But this particularly issue does not have anything to with the story I want to tell.

My Experience with the MRI Conditional Project

The St. Jude Medical MRI conditional pacemaker was engineered to enable patients to undergo an MRI scan. To insure that pacemaker patients would not be harmed by the scan required that the operating settings on the pacemaker be adjusted. (To make a long story short ... a change in the setting needed to make sure that the sensing lead to heart be turned off. The pacemaker could be changed to constant pace or turned off entirely if the patient is not pacemaker dependent ... as most pacemaker patients are.)

So the major problem in this entire issue was in regards to how to change the settings on the device? Who would do it, how would it be done, what would the settings be? Essentially three basic approaches were considered:

1. Have the patient's cardiac physician or cardiac nurse go to the MRI center, lugging their device programmer with them, change the settings on the patient's device to those that are MRI compatible, wait for the scan to complete, reset the settings to normal and examine the patient to insure that the patient is OK.
2. Have the settings changed remotely. The patient is at the MRI center, the cardiac professional is in the office, at the hospital or at home. This is known as "remote programming."  At the time this was something that the FDA did not allow. Using remote programming, the patient's device communicates wireless to a pacemaker communicator located at the MRI center. The cardiac professional sees a 30 second rhythm strip before setting the patient's device to the MRI settings and sees another 30 second rhythm strip after the changes have been made. (Just like an onsite cardiac professional would do.) The patient undergoes the scan. During that time, the professional can perform other tasks. Once the scan is complete, the cardiac profession changes the pacemaker settings back to normal and sees the before and after rhythm strips. 
3. The pacemaker is programmed with two settings by the cardiac professional using the programmer. The first set of settings define the normal operation of the pacemaker. The second set are the MRI settings: that is, the settings of the pacemaker when the patient undergoes an MRI scan.  When the pacemaker patient goes to the MRI center, the MRI tech takes a wand (that's best way I can describe it.) and changes the settings from normal to MRI. Once the patient completes the MRI scan, the MRI tech uses the wand to change the patient's setting back to normal. 

I became quickly apparent that cardiac professionals had no interest in option 1. As it turned out St. Jude Medical chose the third approach. 

When the third approach was described, I had numerous objections ... mostly related to the device that would change the setting on the pacemaker. Thankfully, there have been substantial changes and upgrades made to the wand. However, I wanted to purse option 2, remote programming. And the desire to purse option 2 inspired me to start this blog ... hence the title Medical Monitoring & Remote Programming.

Wherefore Remote Programming?

Most physicians showed some hesitancy when it came to adopting remote programming. They saw it as unproven ... and they were right, it was (and so far as I know still is) unproven and still not acceptable to the FDA. However, many if not most were intrigued by the idea and thought that the technology should be pursued. Many clearly saw the potential value of the technology, the value of being able to monitor patients remotely with the potential ability to change cardiac device settings without the patient being in the office could be a revolution in patient care ... not only for people with chronic conditions like heart problems, diabetes or neurological problems that involve implanted devices, but potentially everyone. And it need not involve the need for implanted or wearable devices. We'll explore this in later postings.

Medical Design Article: FDA announces Medical Device Home use Initiative

As I was working on a human factors related article, this article from Medical Design appeared.  Here's the link to the article: http://medicaldesign.com/contract-manufacturing/fda-announces-medical-device-home-050310/

I thought that this article is interesting and telling with respect to how the FDA will assert it regulatory authority regarding usability issues. Here are a few quote from the article.

Recognizing that more patients of all ages are being discharged from hospitals to continue their medical treatment at home, the U.S. Food and Drug Administration announced an initiative to ensure that caregivers and patients safely use complex medical devices in the home. (My emphasis.) The initiative will develop guidance for manufacturers that intend to market such devices for home use, provide for post-market surveillance, and put in place other measures to encourage safe use of these products. The FDA is also developing education materials on home use of medical devices.

These home care patients often need medical devices and equipment such as hemodialysis equipment to treat kidney failure, wound therapy care, intravenous therapy devices, and ventilators.