In January 2022, Anthony Arenz, a 51-year-old living in Mesa, Ariz., breathed a small sigh of relief.
The Galleri blood test, which screens for 50 types of cancer, hadn’t detected any positive signs.
It would be welcome news to anyone but especially to a firefighter with a 9% greater risk of developing cancer and a 14% greater risk of dying from it than the average person. The Mesa unit had lost two servicemen to cancer in the past 3 years. Both were more than a decade younger than Mr. Arenz.
When the city of Mesa offered additional free screening – including a full-body MRI – to firefighters over 50, Mr. Arenz initially shrugged it off. With a negative Galleri test in hand, he didn’t want to spend more time dwelling on it.
Still, he began to feel a creeping guilt for skipping a test that many of his fallen colleagues hadn’t been offered. He tried to soothe his anxiety with research. A look through the company’s website didn’t set him at ease. According to Grail Bio, a test result of “no cancer signal detected” does not rule out cancer.
Mr. Arenz booked his free MRI.
The results left him heavy: stage I kidney cancer. The Galleri test had missed it.
Mr. Arenz received his free Galleri test through a cancer screening program funded by the city of Mesa. The program is housed at Vincere Cancer Center in Scottsdale, Ariz. Under the leadership of radiation oncologist and Vincere co-owner Vershalee Shukla, MD, the program currently screens first responders in more than 10 Arizona cities at no cost to them.
Vincere began using Galleri shortly after the test launched for consumers in June 2021. Since then, the first responder program has become the largest commercial user of the test in North America.
The Galleri test, which has not yet been approved by the Food and Drug Administration, is so new that few know what incorrect results look like in practice and how often they might occur.
After running the test on about 2,000 servicemen and servicewomen, Dr. Shukla can offer some insight about the test’s real-world value in a high-risk population.
“Cancer screening is a very complicated issue,” Dr. Shukla said in an interview. “Being honest, the tests are good but are not ready yet [for wider use].”
Mr. Arenz was not the only firefighter who got a surprise after taking a Galleri test.
In nearby Phoenix, 51-year-old firefighter Mike Curtis knew his risk for cancer was high, but he wasn’t that worried. Mr. Curtis had been running into fires since he was 17. His dad, also a firefighter, had died of cancer at age 58.
Mr. Curtis had taken the Vincere Cancer Center up on every free screening service since the program began in late 2018 – well before Dr. Shukla started using Galleri in 2021. His most recent lung CT was clear. But he underwent the Galleri test just to stay vigilant.
His result was a shock. The test detected signs of cancer.
Mr. Curtis decided to tell no one, not even his wife. He’d bear the bad news alone until he was certain.
Dr. Shukla, however, immediately doubted the blood test result. She expedited several follow-up tests. One week, a PET, and CT of the abdomen and pelvis later, her hunch was confirmed. The Galleri test result was wrong, Mr. Curtis did not have cancer.
The price of his peace of mind: an extensive workup with a $4,000 price tag. Fortunately, the bill was covered by the screening program.
Overall, in just over 18 months of using the blood test, Dr. Shukla has only encountered 1 other false positive out of about 2,000 Galleri results.
She also discovered two positive signals for cancer using Galleri that were confirmed with follow-up tests. One was a chordoma, a rare type of bone cancer, and the other was a squamous cell carcinoma of the head and neck. The Galleri test caught both remarkably early, in time for treatment.
For Dr. Shukla, however, false negatives were particularly “horrible.” Mr. Arenz’s was just 1 of 28 cancers that the blood test missed. And because 500 negative tests are yet to be validated, the 28 false negatives may be an underestimate.
In her experience, the binary test result – a simple positive or negative cancer signal – is an oversimplification of risk, she said. It “gives a false perception that you have cancer or you don’t,” although the test itself is not definitive.
Grail senior medical director Whitney Jones, MD, agreed that the test is not meant to be a stand-alone screening test for cancer. The purpose of the Galleri test is to “complement other screenings, not replace them,” Dr. Jones told this news organization.
According to an analysis of Galleri data and Dr. Shukla’s experience, the test’s specificity was over 99%. That means the test successfully minimizes false positives.
But the test’s sensitivity was much lower. From data from first responders, Dr. Shukla determined the sensitivity to be 6.7%. That means the test misses about 93 of every 100 cancers. According to Grail’s latest data from more than 6,300 people older than 50, the test’s sensitivity was 29%.
Specificity and sensitivity are metrics used to credential a test and establish confidence in its ability to detect the target disease. A test with high specificity can correctly identify patients who do not have the condition in question, while a test with high sensitivity can correctly identify patients who do have the disease. But there are trade-offs between sensitivity and specificity. One value is increased at the expense of the other.
It’s normal for a cancer screening test to prioritize specificity, according to Aparna Parikh, MD, an oncologist at Mass General Cancer Center in Boston. In a test like Galleri, which is meant to be an adjunct to other screening modalities, “at least we are seeing a good specificity, which is important, because we don’t want false positives, where the downstream impact on the patient can be high.”
Overall, Dr. Jones said, Grail Bio’s aim is to build a test that’s sensitive enough to catch the most dangerous cancers without inundating the healthcare system with false positives. In addition, Dr. Jones explained, sensitivity varies by cancer type. It tends to be lower for cancers for which other screening modalities are available, as well as for earlier-stage disease.
However, the Galleri sensitivity values are “a little bit scary,” said Ji-Hyun Lee, DrPH, professor of biostatistics at the University of Florida and director of the division of quantitative sciences at the University of Florida Health Cancer Center, both in Gainesville. Dr. Lee, who is not affiliated with Grail, reviewed the company’s publicly available data as well as Dr. Shukla’s data at the request of this news organization.
While there’s no definitive threshold for sensitivity, miss rates as high as 93% and 71% “provide little confidence in the [accuracy of the] test,” Dr. Lee said.
Positive and negative predictive values, however, are more clinically relevant measures of a screening test. These numbers indicate how likely it is that a patient’s results are true and therefore how worried they should be about a positive result and how much they should trust a negative result.
Galleri’s data in the over-50 population and Dr. Shukla’s in first responders suggest the test’s negative predictive value is very high – 98.6% and 98.1%, respectively – which means most people can trust a negative test result.