Submission of the Human Engineering File to the FDA and Other Regulatory Bodies, Section 8: Part VI

This is the easiest for me to cover largely because the requirements for the validation section are clearly spelled out in detail.

8

Details of human factors validation testing Rationale for test type selected (i.e., simulated use, actual use or clinical study) Test environment and conditions of use Number and type of test participants Training provided to test participants and how it corresponded to real-world training levels Critical tasks and use scenarios included in testing Definition of successful performance of each test task  Description of data to be collected and methods for documenting observations and interview responses Test results: Observations of task performance and occurrences of use errors, close calls, and use problems  Test results: Feedback from interviews with test participants regarding device use, critical tasks, use errors, and problems (as applicable)   Description and analysis of all use errors and difficulties that could cause harm, root causes of the problems, and implications for additional risk elimination or reduction

These requirements are largely self explanatory. However, I would like to make a few comments and additions.

• Validation testing -- including verification testing -- are often performed by outside consulting firms. Thus it is extremely important that you spell out how your testing should be performed and the measurements to be collected and reported. I've noted that often times consulting company is asked to write both the protocol and the testing script. This is a mistake. The organization that performed the work up to the validation testing stage should be responsible for creating the protocol and the script, because it is this organization that will be responsible for the submission of the HE file to the FDA and/or other regulatory bodies. It's important that the research and development as well as the submission be responsible and in full control of what takes place during the validation step.
• Verification and Validation testing. Verification testing takes place under laboratory conditions using as testing participants members of the targeted user population. This is an additional check on the usability of the system or device. Validation testing takes place in actual or simulated actual conditions -- with all the distractions and problems that users will likely encounter.
• Rationale for type of testing performed and the conditions chosen for validation testing can be extremely important especially if you have chosen a testing procedure less rigorous than performance testing under real or simulated real conditions. Consult IEC 62366 and AAMI HE75 for guidance.
• Testing procedure should insure that full testing of critical tasks are performed and likely to be repeatedly performed by testing participants.
• Suggested additional measurement: If your system or device has error trapping and redirecting capabilities, be sure to report how often these capabilities were triggered and if they enabled the testing participant to successfully complete the task. This could be labeled as: task successfully completed, close call. However, a system or device with the capability to protect against use errors is a capability worth pointing out. 

What to include in your narrative?

Include the abstract or abstracts of your validation testing in your narrative. 

If you haven't included any significant issues or root cause analysis in your abstract, be sure to include this in your narrative. Be sure you surface all issues or concerns in your narrative, if you don't it could appear to a reviewer that you're trying to hide any problems that you encountered. Even the appearance of hiding problems could cause problems with receiving approval for your system or device. 

Apple Watch 5: Heart Monitoring Capabilities -- Afib

The Apple Watch 5 has a heart rhythm monitoring capability that is tuned to detecting the presence of atrial fibrillation, AKA, Afib. Apple categorically states that the watch is unable to detect a heart attack. (And by implication, the likelihood of a heart attack occurring within minutes or hours.)

You have to manually enable your heart monitoring system (Watch and iPhone) to detect Afib. This not part of the default configuration. Here's the link for setting it up: https://support.apple.com/en-us/HT208931#afib

Here's what Apple says about the capabilities of their system and note that it requires both the Apple Watch 5 and an iPhone: 

INDICATIONS FOR USE (NON-EU REGIONS)

The Irregular Rhythm Notification Feature is a software-only mobile medical application that is intended to be used with the Apple Watch. The feature analyzes pulse rate data to identify episodes of irregular heart rhythms suggestive of atrial fibrillation (AF) and provides a notification to the user. The feature is intended for over-the-counter (OTC) use. It is not intended to provide a notification on every episode of irregular rhythm suggestive of AF and the absence of a notification is not intended to indicate no disease process is present; rather the feature is intended to opportunistically surface a notification of possible AF when sufficient data are available for analysis. These data are only captured when the user is still. Along with the user’s risk factors, the feature can be used to supplement the decision for AF screening. The feature is not intended to replace traditional methods of diagnosis or treatment.

The feature has not been tested for and is not intended for use in people under 22 years of age. It is also not intended for use in individuals previously diagnosed with AF.

INTENDED PURPOSE (EU REGION)

Intended Use

The Irregular Rhythm Notification Feature (IRNF) is intended to pre-screen and notify the user of the presence of irregular rhythms suggestive of atrial fibrillation (AF). The feature can be used to supplement a clinician’s decision to screen for possible AF. The feature is intended for over-the-counter (OTC) use.

The feature has not been tested for and is not intended for use in people under 22 years of age. It is also not intended for use in individuals previously diagnosed with AF.

Indications

The feature is indicated to pre-screen for irregular rhythms suggestive of AF for anyone aged 22 years and over.

USING THE IRREGULAR RHYTHM NOTIFICATION FEATURE Set-Up/On-boarding

• Open the Health app on your iPhone.
• Navigate to “Heart”, then select “Irregular Rhythm Notifications”.
• Follow the onscreen instructions.

Receiving a Notification

Once the feature is turned on, you will receive a notification if the feature identified a heart rhythm suggestive of AF and confirmed it on multiple readings.
If you have not been diagnosed with AF by a GP, you should discuss the notification with your doctor.

All data collected and analysed by the Irregular Rhythm Notification Feature is saved to the Health app on your iPhone. If you choose to, you can share that information by exporting your health data in the Health app.

SAFETY AND PERFORMANCE

In a study of 226 participants aged 22 years or older who had received an AF notification while wearing Apple Watch and subsequently wore an electrocardiogram (ECG) patch for approximately one week, 41.6% (94/226) had AF detected by ECG patch. During concurrent wear of Apple Watch and an ECG patch, 57/226 participants received an AF notification. Of those, 78.9% (45/57) showed concordant AF on the ECG patch and 98.2% (56/57) showed AF and other clinically relevant arrhythmias. A total of 370 irregular rhythm notifications with readable ECG patch data was received by the 57 participants. Of those 370 notifications, 322 (87.0%) were assessed to be AF, 47 (12.7%) were arrhythmias other than AF and 1 (0.3%) was sinus rhythm. These results demonstrate that, while in the majority of cases the notification will accurately represent the presence of AF, in some instances, a notification may indicate the presence of an arrhythmia other than AF. No serious device adverse effects were observed.

CAUTIONS

The Irregular Rhythm Notification Feature cannot detect heart attacks. If you ever experience chest pain, pressure, tightness or what you think is a heart attack, call emergency services.

The Irregular Rhythm Notification Feature is not constantly looking for AF and should not be relied on as a continuous monitor. This means the feature cannot detect all instances of AF and people with AF may not get a notification.

• Not intended for use by individuals previously diagnosed with AF.
• Notifications made by this feature are potential findings, not a complete diagnosis of cardiac conditions. All notifications should be reviewed by a medical professional for clinical decision making.
• Apple does not guarantee that you are not experiencing an arrhythmia or other health conditions even in the absence of an irregular rhythm notification. You should notify your GP if you experience any changes to your health.
• For best results, make sure your Apple Watch fits snugly on top of your wrist. The heart rate sensor should stay close to your skin.

From the information provided I am unable to determine how the Afib monitoring system detects Afib. It does seem use an additional capability beyond heart rate system, but from what little I can understand, it uses software running on either the watch and/or the iPhone and uses as input the data from the heart rate system.

I have no idea what algorithms the Apple heart monitoring system is using to detect atrial fibrillation (AF), but if you read the study above, you'll note that apparently, the Apple system has significant false positive rate. Walking through the study, to qualify as a subject for the study, you had to have had a positive indication of AF by the Apple system. That's the one clear message from the study. Another clear message is that both the Apple system and the AF patch can detect heart arrhythmia  other than AF, but what those were is unclear. Unfortunately the way the data is reported does not provide full clarity into the procedure and results. So there's not much more that I can comfortably conclude.

I feel comfortable stating that if you're wearing the Apple Watch and using the AF detection system and you get an AF indication, it's worth your time to get it checked out even knowing full well that the indication is more than likely to be a false positive.

However, high AF false positive rate of nearly 60% is concerning from the standpoint of those who have the Apple AF detection system activated and receive false positive indications. Information like this gets around and users may have tendency to ignore the AF indications when in fact they should be paying attention to them. To curb the possibility that someone ignores an accurately reported AF indication from the Apple system, it would behove Apple to include with the AF notification a check list displayed on the iPhone the walk the user through to determine if in fact this is an AF event.

Update: Future-Market Analysis: Global Patient Monitoring

I'm posting a link to an article that provides some information from the Global Patient Monitoring Marketing study.  Here's the link: Europe Remote Patient Monitoring Market: Strategic Analysis and Opportunity Assessment.  One warning, the article is loaded with embedded ads and links to services that they want to sell you.  Go to the article, you'll see what I mean.  However, it article provides some information about to size and the growth potential for remote monitoring in Europe.