Apple being sued by New York Cardiologist over Atrial Fibrillation Detection in the Apple Watch

I found this interesting and a bit amusing, but it seems that Apple is being sued by Joseph Wiesel, a clinical assistant professor in cardiology at NYU School of Medicine who alleges "... that the tech giant has infringed a patent—generally related to detecting atrial fibrillation by monitoring a pulse—on which Wiesel is the sole named inventor. The accused products are various versions of the Apple Watch, Series 3 and 4 through purported inclusion of an irregular pulse notification feature, and earlier versions through the alleged provision of a software upgrade to add 'irregular pulse notifications resulting from checking a pulse rhythm'."

Here's a link to the quoted material: https://insight.rpxcorp.com/news/59822?utm_campaign=weekly_newsletter&utm_content=&utm_medium=email&utm_source=title_click

Knowing Apple, they will do everything that they can to invalidate Wiesel's patent. This is a common practice for very large and domineering companies like Apple to do in order to refrain from playing royalties to patent holders, especially when the patent holder is an individual or a small company. 

The processes that have been put in place to examine patents to determine their validity when there is litigation have shown themselves to be quite favorable to large companies being sued for patent infringement. So I suggest that the likelihood that Dr. Wiesel will receive anything from his suit is not all that favorable.

Signal Detection and the Apple Watch

In the last two articles about the Apple Watch's capability to detect atrial fibrillation, I made references to terminology ("false positive") that has its roots in Signal Detection Theory.  Signal Detection Theory was developed as a means to determine the accuracy of early radar systems. The technique has migrated to communications systems, psychology, diagnostics and a variety of other domains where determining the presence or absence of something of interest is important especially when the signal to be detected would be presented within a noisy environment (this was particularly true of  early radars) or when the signal is weak and difficult to detect.  

Signal detection can be powerful tool to guide research methodologies and data analysis. I have used the signal detection paradigm in my own research both for the development of my research methodology and data analysis: planned and post-hoc analysis. In fact when I have taught courses in research methods and statistical analysis, I have used the signal detection paradigm as a way to convey detecting the effects of an experimental manipulation in your data.  

Because I've mentioned issues related to signal detection and that it is a powerful tool for research and development, I decided to provide a short primer of signal detection.

Signal Detection

The central feature of signal detection is the two by two matrix shown below.

The signal detection process begins with a detection window or event. The window for detection could be a period of time or a specified occurrence such as a psychological test such as a rapid presentation of a stimulus and determine whether or not the subject of the experiment detected what was presented. 

Or in the case of the Apple Watch, whether it detects atrial fibrillation. In devices such as the Apple Watch, how the system defines the detection window can be important. Since we have no information regarding how the Apple Watch atrial fibrillation detection system operates, it's difficult to determine how it determines its detection window.

Multiple, Repeated Trials

Before discussing the meaning of the Signal Detection Matrix, it's important to understand that every matrix comes with multiple, repeated trials with a particular detection system, whether that detection system is a machine or a biological entity such as a person. Signal Detection Theory is grounded in probability theory, therefore, there is the requirement for multiple trials in order to create a viable and valid matrix.

The Four Cells of the Signal Detection Matrix

During the window of detection, a signal may or may not be present. Each cell represents an outcome of a detection event. The possible outcomes are: 1: the signal was present and it was detected, a hit (upper left cell), 2: the signal was not present and the system or person correctly correctly reported no signal present (lower right cell), 3: the signal was absent, but erroneously reported as present, this is a Type I error (lower left cell) and 4: the signal was present, but reported as absent, this is a Type II error (upper right cell).

The object of any system is that the outcomes of detection events end up in outcome cells 1 and 2, that is, correctly reported. However, from a research standpoint, the error cells (Outcomes 3 and 4) are the most interesting and revealing. 

Incorrect Report: Cells

Outcome 3: Type I Error

A Type I error is reporting that a signal is present when it was not. This is known as a "false alarm or false positive." The statistic for alpha which is the ratio of Outcome 3 over Total number of trials or detection events.

Outcome 4: Type II Error

A Type II error is reporting that a signal is not present when in fact it was present. This is a "failure to detect." The statistic for beta which is the ratio of Outcome 4 over Total number of trials or detection events. 


If you're designing a detection system, the idea is to minimize both types of errors. However, no system is perfect and as such, it's important to determine what type of error is most acceptable, Type I or II because there are likely to be consequences either way. 

Trade-off Between Type I and Type II Errors

In experimental research the emphasis has largely been on minimizing Type I errors, that is reporting an experimental effect when in actuality none was present. Increasing your alpha level, that is decreasing your acceptance of Type I errors, increases the likelihood of making a Type II error, reporting that an experimental effect was not present when in fact it was. 

However, with medical devices, what type of error is of greater concern, Type I or Type II? That's a decision that will need to be made.

Before leaving this section, I should mention that the trade-off analysis between Type I and Type II errors is called Receiver-Operating-Characteristic Analysis or ROC-analysis. This is something that I'll discuss in a later article. 

With Respect to the Apple Watch 

Since I have no access into Apple's thinking when it was designing the Watch's atrial fibrillation software system, I can't know for certain the thinking that went into designing atrial fibrillation detection algorithm for the Apple Watch. However based on their own research, it seems that Apple made the decision to side on accepting false positives over false negatives -- although we can't be completely sure this is true because Apple did not do the research to determine rate that the Apple Watch failed to detect atrial fibrillation when it was know to be present.

With a "medical device" such as the Apple Watch, it would seem reasonable to side on accepting false positives over false positive. That is, to set your alpha level low. The hope would be that if the Apple Watch detected atrial fibrillation the owner of the watch would seek medical attention to determine whether or not a diagnosis of atrial fibrillation was warranted for receiving treatment for the condition. If the watch generated a false alarm, then there was no harm in seeking medical advice ... it would seem. The author of the NY Times article I cited in the previous article appears to hold to this point of view. 

However ...

The problem with a system that generates a high rate of false alarms, is that all too often signals tend to be ignored. Consider the following scenario: an owner of an Apple Watch receives an indication that atrial fibrillation has been detected. The owner goes to a physician who reports that there's no indication of atrial fibrillation. Time passes and the watch reports again that atrial fibrillation has been detected. The owner goes back to the physician who give the owner the same report as before, no atrial fibrillation detected. What do you think will happen if the owner receives from the watch that atrial fibrillation has been detected? It's likely that the owner will just ignore the report. That would really be a problem for the owner if the owner had in fact developed atrial fibrillation. In this scenario the watch "cried wolf" too many times. And therein lies the problem with having a system that's adjusted to accepting a high rate of false alarms.

Follow-up: Apple Watch 5, Afib detection, NY Times Article

The New York Times has published an article regarding the Apple Watch 5's capability to detect atrial fibrillation. The link to the article is below:

https://www.nytimes.com/2019/12/26/upshot/apple-watch-atrial-fibrillation.html?te=1&nl=personal-tech&emc=edit_ct_20191226?campaign_id=38&instance_id=14801&segment_id=19884&user_id=d7e858ffd01b131c28733046812ca088&regi_id=6759438320191226

The title and the subtitle of the article provide a good summary of what the author (Aaron E. Carroll) found:

"The Watch Is Smart, but It Can’t Replace Your Doctor
Apple has been advertising its watch’s ability to detect atrial fibrillation. The reality doesn’t quite live up to the promise."

With reference to my article, the Times article provides more detail on the trial that Apple ran to test the effectiveness of the Apple Watch's ability to detect atrial fibrillation. That provide interesting and enlightening, and clarified some of the issues I found with how the study was reported for both the procedure and the results. In addition, the author and I concur regarding the Apple Watch's extremely high reported rate of false positives for atrial fibrillation. I find this quite interesting when you consider that screening for atrial fibrillation can be as simple as taking the patient's pulse. 

Here are a few quotes from the article:

"Of the 450 participants [these are study participants where the Apple Watch had detected atrial fibrillation] who returned patches , atrial fibrillation was confirmed in 34 percent, or 153 people. 
...

Many news outlets reporting on the study mentioned a topline result: a “positive predictive value” of 84 percent. That statistic refers to the chance that someone actually has the condition if he or she gets a positive test result.

But this result wasn’t calculated from any of the numbers above. It specifically refers to the subset of patients who had an irregular pulse notification while wearing their confirmatory patch. That’s a very small minority of participants. Of the 86 who got a notification while wearing a patch, 72 had confirmed evidence of atrial fibrillation. (Dividing 72 by 86 yields 0.84, which is how you get a positive predictive value of 84 percent.)

Positive predictive values, although useful when talking to patients, are not always a good measure of a test’s effectiveness. When you test a device on a group where everyone has a disease, for instance, all positive results are correct."
...

There are positive messages from this study. There’s potential to use commercial devices to monitor and assess people outside of the clinical setting, and there’s clearly an appetite for it as well. But for now and based on these results, while there may be reasons to own an Apple Watch, using it as a widespread screen for atrial fibrillation probably isn’t one."

Apple Watch 5: Heart Monitoring Capabilities -- Afib

The Apple Watch 5 has a heart rhythm monitoring capability that is tuned to detecting the presence of atrial fibrillation, AKA, Afib. Apple categorically states that the watch is unable to detect a heart attack. (And by implication, the likelihood of a heart attack occurring within minutes or hours.)

You have to manually enable your heart monitoring system (Watch and iPhone) to detect Afib. This not part of the default configuration. Here's the link for setting it up: https://support.apple.com/en-us/HT208931#afib

Here's what Apple says about the capabilities of their system and note that it requires both the Apple Watch 5 and an iPhone: 

INDICATIONS FOR USE (NON-EU REGIONS)

The Irregular Rhythm Notification Feature is a software-only mobile medical application that is intended to be used with the Apple Watch. The feature analyzes pulse rate data to identify episodes of irregular heart rhythms suggestive of atrial fibrillation (AF) and provides a notification to the user. The feature is intended for over-the-counter (OTC) use. It is not intended to provide a notification on every episode of irregular rhythm suggestive of AF and the absence of a notification is not intended to indicate no disease process is present; rather the feature is intended to opportunistically surface a notification of possible AF when sufficient data are available for analysis. These data are only captured when the user is still. Along with the user’s risk factors, the feature can be used to supplement the decision for AF screening. The feature is not intended to replace traditional methods of diagnosis or treatment.

The feature has not been tested for and is not intended for use in people under 22 years of age. It is also not intended for use in individuals previously diagnosed with AF.

INTENDED PURPOSE (EU REGION)

Intended Use

The Irregular Rhythm Notification Feature (IRNF) is intended to pre-screen and notify the user of the presence of irregular rhythms suggestive of atrial fibrillation (AF). The feature can be used to supplement a clinician’s decision to screen for possible AF. The feature is intended for over-the-counter (OTC) use.

The feature has not been tested for and is not intended for use in people under 22 years of age. It is also not intended for use in individuals previously diagnosed with AF.

Indications

The feature is indicated to pre-screen for irregular rhythms suggestive of AF for anyone aged 22 years and over.

USING THE IRREGULAR RHYTHM NOTIFICATION FEATURE Set-Up/On-boarding

• Open the Health app on your iPhone.
• Navigate to “Heart”, then select “Irregular Rhythm Notifications”.
• Follow the onscreen instructions.

Receiving a Notification

Once the feature is turned on, you will receive a notification if the feature identified a heart rhythm suggestive of AF and confirmed it on multiple readings.
If you have not been diagnosed with AF by a GP, you should discuss the notification with your doctor.

All data collected and analysed by the Irregular Rhythm Notification Feature is saved to the Health app on your iPhone. If you choose to, you can share that information by exporting your health data in the Health app.

SAFETY AND PERFORMANCE

In a study of 226 participants aged 22 years or older who had received an AF notification while wearing Apple Watch and subsequently wore an electrocardiogram (ECG) patch for approximately one week, 41.6% (94/226) had AF detected by ECG patch. During concurrent wear of Apple Watch and an ECG patch, 57/226 participants received an AF notification. Of those, 78.9% (45/57) showed concordant AF on the ECG patch and 98.2% (56/57) showed AF and other clinically relevant arrhythmias. A total of 370 irregular rhythm notifications with readable ECG patch data was received by the 57 participants. Of those 370 notifications, 322 (87.0%) were assessed to be AF, 47 (12.7%) were arrhythmias other than AF and 1 (0.3%) was sinus rhythm. These results demonstrate that, while in the majority of cases the notification will accurately represent the presence of AF, in some instances, a notification may indicate the presence of an arrhythmia other than AF. No serious device adverse effects were observed.

CAUTIONS

The Irregular Rhythm Notification Feature cannot detect heart attacks. If you ever experience chest pain, pressure, tightness or what you think is a heart attack, call emergency services.

The Irregular Rhythm Notification Feature is not constantly looking for AF and should not be relied on as a continuous monitor. This means the feature cannot detect all instances of AF and people with AF may not get a notification.

• Not intended for use by individuals previously diagnosed with AF.
• Notifications made by this feature are potential findings, not a complete diagnosis of cardiac conditions. All notifications should be reviewed by a medical professional for clinical decision making.
• Apple does not guarantee that you are not experiencing an arrhythmia or other health conditions even in the absence of an irregular rhythm notification. You should notify your GP if you experience any changes to your health.
• For best results, make sure your Apple Watch fits snugly on top of your wrist. The heart rate sensor should stay close to your skin.

From the information provided I am unable to determine how the Afib monitoring system detects Afib. It does seem use an additional capability beyond heart rate system, but from what little I can understand, it uses software running on either the watch and/or the iPhone and uses as input the data from the heart rate system.

I have no idea what algorithms the Apple heart monitoring system is using to detect atrial fibrillation (AF), but if you read the study above, you'll note that apparently, the Apple system has significant false positive rate. Walking through the study, to qualify as a subject for the study, you had to have had a positive indication of AF by the Apple system. That's the one clear message from the study. Another clear message is that both the Apple system and the AF patch can detect heart arrhythmia  other than AF, but what those were is unclear. Unfortunately the way the data is reported does not provide full clarity into the procedure and results. So there's not much more that I can comfortably conclude.

I feel comfortable stating that if you're wearing the Apple Watch and using the AF detection system and you get an AF indication, it's worth your time to get it checked out even knowing full well that the indication is more than likely to be a false positive.

However, high AF false positive rate of nearly 60% is concerning from the standpoint of those who have the Apple AF detection system activated and receive false positive indications. Information like this gets around and users may have tendency to ignore the AF indications when in fact they should be paying attention to them. To curb the possibility that someone ignores an accurately reported AF indication from the Apple system, it would behove Apple to include with the AF notification a check list displayed on the iPhone the walk the user through to determine if in fact this is an AF event.