using surrogate markers for disease: are they really appropriate?

 An important article (i think) just came out that evaluated the robustness of surrogate markers as accurate substitutes for clinical outcomes, finding many concerns (see surrogate markers concerns JAMA2024 in dropbox, or doi:10.1001/jama.2024.4175)

 

Details:

-- researchers accessed the FDA Adult Surrogate Endpoint Table (with >100 entries) and MEDLINE from the inception of this FDA table in 2018 to March 2023, evaluating the correlation between the surrogate markers and clinical endpoints found in the relevant meta-analyses (though they excluded oncologic treatments)

-- 37 surrogate markers were identified for 32 unique nononcologic chronic diseases

  

Results (my comments are embedded in the review of many of the individual surrogate markers below):

-- for 22 (59%) of the surrogate markers in 21 chronic diseases, no eligible meta-analysis was identified

    -- 17 (77%) of these surrogate markers were classified by the FDA as being appropriate for traditional approval (eg FEV₁ for asthma and urine toxicology for opioid use disorder)

    -- 5 (23%) were deemed appropriate for accelerated approval

-- for 15 (41%) of the surrogate markers that could be used as primary endpoints in 14 chronic diseases, at least 1 meta-analysis was identified, with median number of trials of 18.5 and 90,056 patients analyzed

    – of these 15 surrogate markers, 13 were classified as appropriate for traditional approval and 2 for accelerated approval

-- of the 54 meta-analyses reported with 109 unique surrogate marker-clinical outcome pairs:

    --59 (54%) reported at least one correlation coefficient (r or R²):

        -- 10 (17%) reported at least 1 outcome classified as "high strength evidence", per the PRISMA guidelines (Preferred Reporting Items for Systematic Reviews and Meta-analyses)

        -- 50 (46%) reported just slopes, effect estimates, or results of meta-regression analyses only

        -- 26 (52%) reported at least 1 statistically significant result

 

Surrogate markers that may be used for accelerated approval:

-- Alzheimer's disease, using the surrogate marker of reduction in beta-amyloid plaques:

    -- 3 meta-analyses: one had low-strength correlation coefficients between reductions in beta-amyloid deposition and clinical changes in the Clinical Dementia Rating Scale (CDRS), another found a decrease in beta-amyloid levels by PET scan but that was not associated with a reduction in CDRS, and a third had a 0.1 unit decrease in beta-amyloid and a 0.09 point decrease in the CDRS (a scale from 0-18, with a clinically meaningful difference being 0.5 points: ie not clinically meaningful), and a 0.13 point change in the MMSE (scale 0-30, with 1-3 points being clinically meaningful: ie not clinically meaningful).

    -- my comments: 

        -- there were quite conflicting findings on the role of beta-amyloid plaques in the pathophysiology of Alzheimer's. 

        -- in 2021 the FDA approved the remarkably expensive drug aducanumab (adulhelm) at $56,000/yr per person for potentially lots of people, the cost of which was in line to destroy the Medicare budget. This approval was based on the drug's ability to reduce brain amyloid plaque. At the time of approval, there was major controversy about the importance of amyloid plaques as a driver for developing or reversing Alzheimer's, major journals came out strongly against its approval (eg https://pubmed.ncbi.nlm.nih.gov/35666531/ ), the FDA panel assessed its utility and voted "NO" to approval, but the FDA still gave approved it, ironically as an accelerated approval. Medicare (very untypically) actually voted against covering this med: there were significant concerns about adverse effects (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10002282/), concerns about its efficacy (https://www.jpreventionalzheimer.com/5919-two-randomized-phase-3-studies-of-aducanumab-in-early-alzheimers-disease.html ), and the drug was pulled from the market by Biogen, its manufacturer, in 2024

        -- so, choosing to use beta-amyloid plaque reduction as the target of drug efficacy, as opposed to a real clinical outcome (progressive dementia) is really pretty absurd

 

-- Primary glomerular disease, using the surrogate marker of proteinuria:

    --1 meta-analysis had high-strength evidence of proteinuria being associated with the composite endpoint of doubling serum creatinine level, end-stage kidney disease, or death

    -- my comments:

        -- i checked out this meta-analysis: it was limited to those with IgA nephropathy and not other primarily glomerular diseases (see https://www.ajkd.org/article/S0272-6386(16)00214-6/abstract )!!! 

        -- there are some older studies (eg REIN: Ramipril Efficacy in Nephropathy), that found that non-diabetic patients with nephrotic syndrome and proteinuria of at least 3 grams/d had benefit from ramipril in terms of decreasing renal decline, there was some but less benefit in those with 1-3g/d, and much less in patients with 0.5-1.0 gram (eg see https://pubmed.ncbi.nlm.nih.gov/9407422/ ). this is in pretty stark contrast to diabetic patients, where much lower levels of proteinuria (albumin excretion of >30 mg/g a day) should be treated with an ACE-inhibitor for example in order to prevent renal function decline. 

        --so, there are concerns about generalizing the specifics of proteinuria as a surrogate marker found with IgA nephropathy to the many causes of primary glomerular disease

 

Surrogate markers that may be used for traditional approval (i will include their analysis for the most clinically relevant ones where there was at least one meta-analysis):

-- chronic kidney disease, using eGFR as the surrogate marker:

    --2 meta-analyses, one with high-strength coefficient of determination between eGFR and the composite endpoint of doubling of serum creatinine level, eGFR<15, and treated end-stage kidney disease; another showed that a 30% and 40% reduction in eGFRs were associated with statistically significant improvements in treating end-stage kidney disease

    -- my comments: 

        -- the first meta-analysis they mention used eGFR as both the subject and part of the composite endpoints, which is not a strong evaluation of the role of eGFR as an independent marker of CKD.... 

        -- most studies in the US use creatinine-based eGFR measurements, which has been found to be less predictive of the actual measured GFR than cystatin-C based eGFR (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6961921/, https://www.kidney.org/atoz/content/cystatinC)

        -- cystatin-C has been found to be a superior risk predictor of all-cause mortality vs creatinine in a Manhattan study with a diverse population (eg https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0226509 , 

        -- and, cystatin-C based eGFR, when significantly lower than the creatinine-based eGFR (which happens very frequently as noted in the literature as well as in my pretty extensive experience of comparative testing), is much more predictive of future renal and cardiovascular adverse events: https://gmodestmedblogs.blogspot.com/2023/12/cystatin-c-better-predictor-of-bad.html

 

-- COPD, using FEV₁ as the surrogate marker:

    -- 6 meta-analyses but with inconsistent evidence of the strength of association between improvements in FEV₁ by both objective and subjective clinical outcomes; only 2 or more high-strength coefficients between change in trough FEV₁ and first occurrence of a moderate to severe exacerbation across all trials as well as in those evaluating only bronchodilators. This use of FEV₁, per their evaluation, only applies to those with known COPD, noting that a change in FEV₁ was associated with changes in subjective and objective clinical outcomes. and high-strength evidence only identified a few high-strength associations

    -- my comments:

       -- though the point in the FDA approval was assessing FEV₁ as the surrogate marker for COPD, these meta-analyses used for that assessment actually studied changes in FEV₁ and not for defining COPD

            -- it is  well-known that asymptomatic people (without known respiratory problems) can have normal FEV₁ assessments: https://rc.rcjournal.com/content/64/5/570.  so, one cannot extend the FEV₁ surrogate marker to those without a clear history of COPD

        -- and, the usual assessment for COPD involves assessing FVC as well as FEV₁ by spirometry

 

--Gout, using serum uric acid level as the surrogate marker: 

    --2 meta-analyses with inconsistent evidence of the strength of the association between serum uric acid and various clinical evidence. one found low-strength association between the achieved serum urate level of <6mg/dL and gout flairs; another found statistically significant slopes, noting serum urate reductions were actually associated with more pain and patient-reported outcomes

    – my comments:

        -- this last meta-analysis finding increased pain reported results only in the first 6 months of therapy, and it is well-known that there is no real decrease in gout attacks until after 6 months (ie, i don't think this meta-analysis was particularly useful)

        -- higher uric acid levels do seem to be associated with increased cardiovascular events (uric acid threshold for increased CAD mortality (gmodestmedblogs.blogspot.com)), and decreasing serum uric acid seems to be associated with decreased cardiovascular events (allopurinol decreases heart disease?? (gmodestmedblogs.blogspot.com) 

        -- another issue: chronic colchicine use, which also decreases gout attacks but does not lower uric acid levels, is a pretty strong anti-inflammatory, even in patients without gout but a history of cardiovascular events: colchicine decreases cardiovasc events if diabetes and recent MI (gmodestmedblogs.blogspot.com), and also in those with gout (colchicine may lower cardiac risk in patients with gout (gmodestmedblogs.blogspot.com). it would be great to have a study assessing both gouty attacks and cardiovascular outcomes in patients randomized to allopurinol vs colchicine...

 

--HIV, using plasma HIV RNA as the surrogate marker:

    -- 2 meta-analyses but with inconsistent evidence on the strength of association between HIV-1 RNA viral load and progression to AIDS or death; one reported that increased HIV1-RNA was associated with higher rates of progression to AIDS; the other that there was high-strength coefficient of determination that viral load <200 (but not if <50) copies/mL was associated with progression to AIDS or death at 48 weeks 

    --my comments: 

        -- there can be a divergence between effective viral load suppression by meds and improvement in the immunologic markers (ie, we have phenomenal drugs to treat the former and nothing much for the latter): for one pretty extreme example, i have a 60yo woman who has consistently had a non-detectable viral load on meds for the past 10 years, though she has never had a CD4 count above 124. She was diagnosed with metastatic Kaposi's sarcoma 4 years ago which is currently advanced but pretty stable on chemotherapy, as well as progressive vaginal, cervical, and anal dysplasia for the past 7 years. She continues to have recurrent oral candidasis responsive to meds and a several episodes bacterial pneumonias (ie, excellent viral load response to meds, but continues to have many AIDS-associated outcomes)

        -- one study found that patients with undetectable viral loads, median CD4 level 510 (but 10% had between 200-350), still had a 34% increased risk of hospitalizations, especially for AIDS-defining illness, Ob/Gyn, and hematologic problems: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4447832/ 

        -- another study assessed hospitalization rates and found that elite controllers (the small group of people who have undetectable HIV viral loads but do have HIV antibodies and delayed or no progression of HIV, without taking meds) had higher hospitalization rates than those who non-elite controllers who had a suppressed viral load on meds, and both were higher than that of matched non-infected controls but mostly in non-AIDS defining conditions (cardiovascular and psychiatric): https://academic.oup.com/jid/article/211/11/1692/858186 .  ??role of higher CRP levels (reflecting inflammation) in the elite controllers and also in those with controlled HIV by meds

        -- so this situation highlights that surrogate markers may be more useful for some aspects of a disease than others: it is quite clear that an undetecgtable viral load diminishes the chances of HIV transmission, but may be less robust for some of the other associated conditions

 

 -- Lipids, using LDL as the surrogate marker:

    -- 11 meta-analyses finding inconsistent evidence on the strength of association between reduction in LDL and various clinical outcomes; all found a significant improvement in at least on 1 clinical outcome but only 1 had high-strength coefficients of determination between LDL and major vascular events, major coronary events, and vascular mortality in trials investigating statin therapy

    -- my comments:

         -- as mentioned in many prior blogs, there are many lipid particles that migrate electrophoretically in the  arbitrarily-defined "low density" region. Small, dense LDLs are much more atherogenic than the larger ones, and this is likely related to their increased oxidizability and binding to tissue macrophages and creating early atherosclerotic changes (in this regard, measuring apoliporotein B levels, as recommended in the European cardiology guidelines, is likely a better marker, since it differentiates LDL size better: CAD: presumed mechanism and apoB was best predictor of CAD, by mendelian randomization (gmodestmedblogs.blogspot.com)

        -- so LDL is an off-target surrogate marker that works well epidemiologically for large populations of patients overall, but may not be so accurate in an individual patient

 

-- Lipids, using high triglycerides (TG) as the surrogate marker:

    – 3 meta-analyses had inconsistent evidence or relationship between high triglycerides and several clinical events.

   -- my comments:

        -- there is debate about the specific role of high TG in causing cardiovascular events. high TG levels are associated with low HDL levels and high LDL levels. And TG are associated with other potential cardiovascular risk factors, including apoB levels, Factor VII, obesity, metabolic syndrome and diabetes, CKD, nephrotic syndrome, endothelial dysfunction. And the studies linking high TG to cardiovascular events have not controlled for all of these associations, which in and of themselves might be associated with the noted increase in cardiovascular events in individuals with high TGs in some studies 

        -- the Quebec study found that triglycerides were associated with LDL size: if the TG level was <132 mg/DL, the highly atherogenic small, dense LDL particles were very unlikely; if >176 mg/dL, small, dense LDLs were very likely

        -- another study found a pretty dramatic relationship between the TG/HDL radio and the likelihood of small, dense LDLs, with a ratio of >3.8 having 78% predictive value for the small, dense ones; and <3.8 having an 81% likelihood of the less atherogenic large cholesterols (Hanak V, et al. Am J Cardiol. 2004 Jul 15;94(2):219-22.doi: 10.1016/j.amjcard.2004.03.069)

        -- so, in those studies finding a high TG/cardiovascular disease relationship, is it the high TGs? or any one or combination of the cardiovascular risk factors associated with high TGs????

        

-- osteoporosis, using bone mineral density (BMD) as the surrogate marker:

    --6 meta-analyses finding inconsistent evidence on the strength of association between increases in BMD (hip, femoral neck, spine) with reductions in the risk of related fractures.  the 2 most recent meta-analyses  found that 3 of the 18 coefficients of determination were high-strength, specifically hip BMD and vertebral fractures, femoral neck BMD and nonvertebral fractures, and spine BMD and vertebral fractures

    --my comments:

        --10-20% of "osteoporosis-related " fractures" are in people who have normal BMD by DXA, and most people who do have fractures have BMDs in the sub-osteoporosis range (see fracture with normal BMD ArchOsteopor2020 in dropbox or doi.org/10.1007/s11657-020-00745-0), likely related to problems with "bone quality".

        --BMD is a pretty gross radiologic assessment of the amount of calcium across the diameter of the bone assessed. there is no assessment of the actual quality of the microarchitecture of the bone (eg trabecular bone thickness, structure or connectivity, or cortical bone thickness and porosity) or inclusion of bone turnover markers in the routine calculus of osteoporosis (as with the FRAX score) or in the approach to routine clinical identification and management of osteoporosis. one prescient example of this is in a few older trials of the use of high dose fluoride in treatment of osteoporosis: large increases in BMD but increased fracture risk attributable to denser, but more brittle bone (there seems to be increased bone volume and trabecular thickness but no increase in trabecular connectivity): https://www.nejm.org/doi/full/10.1056/NEJM199003223221203. unfortunately, there are no clinically available measures of bone quality/integrity that might be important as surrogate markers.

        -- we do have good markers for bone turnover (CTX, NTX, bone-specific alkaline phosphatase), and elevations of these have been found to be associated with fracture risk, but no studies i have found that compares them to BMD, or assesses the fracture risk utilizing both BMD and marker of bone turnover in the predictive model

 

-- Diabetes, using hemoglobin A1c as surrogate marker:

    -- 9 meta-analyses with inconsistent evidence of the strength of association between A1c levels and improvements in all-cause mortality, MI, stroke, heart failure, kidney injury, cardiovascular death, hospitalization for heart failure, coronary heart disease, treatment-related discontinuations, neuropathy, and peripheral vascular events; one meta-analysis had high-strength evidence for fatal stroke and two had high-strength of eviddence for outcome of major adverse cardivascular events

    -- my comments:

        -- the main cause of death in patients with diabetes is cardiovascular

        -- the basis of choosing an A1c of 6.5% to define “diabetes” is that this is the inflection point where retinopathy just begins to increase (https://gmodestmedblogs.blogspot.com/2023/10/update-ascvd-risk-factor-critique.html )

        -- diabetes medication selection: it is likely more important to prevent cardiovascular disease by using medications known to decrease this adverse cardiovascular outcome (GLP1 agonists and/or SGLT2 inhibitors) vs meds that seem to increase this outcome (insulin and sulfonylureas)

        -- the use of the 6.5% A1c cutpoint as defining diabetes really makes no sense, since people with diabetes mostly have major morbidity/mortality from cardiovascular causes (vs retinopathy), and there are several observational studies finding an increased clinical cardiovascular disease beginning with an A1c of about 5.5%

        -- the concern with A1c as a surrogate marker for diabetes is that it might well lull us clinicians and patients into thinking that an A1c<6.5% is good (ie, not diabetes), and perhaps that the choice of anti-diabetic meds or the importance of diet/exercise/having appropriate body weight does not matter (just the A1c achieved)

 

Surrogate markers that may be used for traditional approval but without any meta-analyses of clinical trials:

-- these included FEV₁ for asthma, creatinine for CKD, free cortisol for cushings, fecal coefficient of fat absorption for exocrine pancreatic insufficiency, undetectable plasma HBV DNA for indefinite treatment or using loss of HbsAg for finite treatment for those with hep B infection, sustained viral response by HCV RNA for cure of hep C, serum HIV antibody and >0.5 log reduction in HIV RNA for patients at high risk of sexually-acquired HIV-1, serum TSH level to treat those with hypothyroidism, use of urine proteinuria and eGFR for patients with active lupus nephritis, urine tox screen for those with opioid use disorder, new morphometric vertebral fractures for post-menopausal women with osteoporosis, serum alkaline phosphatase for those wth paget disease, serum calcium level for those with hypercalcemia from primary hyperparathyroidism, FVC for those with systemic sclerosis-interstitial lung disease, or exhaled carbon monoxide level for cigarette smokers.

 

Commentary (see my specific comments on the major common surrogate markers embedded above):

-- surrogate markers are, of course, important in simplifying and expediting the use of new meds as well as for epidemiologic studies assessing medical conditions (eg, using A1c as a marker of diabetes allows an easy and accessible diabetes surrogate in assessing both new meds but also assessing nonpharmacologic benefits, or epidemiological studies of diabetes outcomes)

-- and, surrogate markers have been embraced by the FDA, Public Health departments, clinicians, patients, etc as the appropriate endpoint (eg, is the BMD getting better or worse on meds?)

    -- the FDA Table of surrogate markers "was designed to serve as a reference for drug developers...it does not provide justification for surrogate selection in terms of strength of evidence of associations between the treatment effects measured using the surrogate marker and those measured using disease-relevant clinical outcomes"

    -- this current study was prompted because no studies previously had examined the association for surrogate markers used as primary end points in clinical trials

    -- the researchers assessed meta-analyses since the FDA "maintained that the use of surrogate markers in traditional approval requires, at a minimum, evidence from meta-analyses of clinical trials demonstrating an association between surrogate markers and clinical outcomes to establish surrogacy"

-- but, this study shows that large numbers of our accepted surrogate markers had not been tested rigorously and lacked high-strength evidence of associations with clinical outcomes, based on published meta-analyses, and many have significant divergences from some of the really important clinical outcomes for the patients we are treating (and many did not even have the FDA-required meta-analyses!!)

so, what does this all mean??

-- it is abundantly clear to me that these surrogate markers do have significant legitimacy, and are certainly useful.  BMD is a strong predictor of fracture risk. HIV viral load suppression does basically nullify viral transmission. uric acid level is a great target for preventing further gout attacks. A1c levels are important for decreasing the potential ravages of diabetes, etc

-- but it is also clear from this study that these markers are not well-vetted by strong clinical evidence, with the potential of several problems:

    -- these markers may not be actually the real or most important correlates of a clinical disease.

        -- many people with gout do just fine with chronic colchicine and seem to have important cardiovascular protection, as they do with allopurinol. unclear to me from my checking the medical literature if one is better than the other. both have significant adverse effects

        -- BMD is a pretty gross estimation of calcium content of bone, with higher recordings in those with compression fractures, for example; and it does not provide information about the quality of the microarchitecture of the bone, which is important for the bone's structural support. perhaps assessing bone turnover (eg, by CTX) by itself or with BMD is a better predictor

        -- A1c levels can be misleading: they can be inaccurate (patients with hemolytic anemia or other causes of short RBC half-life, or some patients with interference of glycation of RBCs). And there may be a sense of false security to us and to patients if the level is low: even A1c levels as low as 5.5% may be associated with increased cardiovascular risk

        -- a suppressed HIV viral load does not eliminate some of the ravages of HIV: there is still a level of chronic inflammation, which seems to increase cardiovascular risk. there may well be increased incidence of several cancers (a large veteran's study: Park LS Ann Intern Med. 2018;169:87-96. doi:10.7326/M16-2094)

-- in some cases even accelerated approval of  surrogate marker (eg b-amyloid plaques for Alzheimer's disease) may be a huge and ultimately erroneous stretch

-- the real concern overall to me is that both us clinicians and our patients may be lulled into thinking that all is okay with a disease if the surrogate marker is okay

    -- achieving a low LDL on a statin should not mean that there is no problem eating huge amounts of red meat, as has been found in some studies (ie, nonpharmacologic approaches, such as diet and exercise, are essential for preventing lots of diseases, not just heart disease)

    -- there should still be a heightened concern in those with controlled HIV around heart disease (ie, consider more aggressive lipid lowering), cancer (ie generally increased surveillance, as is recommended in  more frequent cervical cancer screenings and without an age to stop)

    -- using A1c as a surrogate marker for diabetes reinforces to us and patients that all is okay if the A1c is under 7%, or even under 6.5%. But, there is still a major increase in cardiovascular disease overall in patients with A1c in the range of 5.5-6.5%, and the meds chosen to treat the diabetes have a major role in the cardiovascular outcomes

    -- and a high eGFR calculated based on creatinine may lull us into unjustified complacency if the more accurate cystatin-based one reveals a much lower eGFR: there are problems both in terms of aggressively assessing and treating the more accurate eGFR and in terms of renal dose-adjusting meds when indicated

-- so the issue of surrogate markers to me is that they are useful in defining broad strokes about clinical diseases for populations, but less useful for a clinical encounter with an individual patient. there are intrinsic problems with their derivation (were the surrogate markers determined by a specific effect on a disease in a specific population?? And perhaps not generalizable to other populations??), many surrogate markers have very poor quality meta-analyses documenting much benefit. And 15 of the identified surrogate markers did not even have one meta-analysis of clinical trials, contrary to the FDA's own criteria requiring a surrogate marker to have "at a minimum, evidence from meta-analyses of clinical trials" to be accepted.....

geoff

-----------------------------------

If you would like to be on the regular email list for upcoming blogs, please contact me at gmodest@bidmc.harvard.edu  

to get access to all of the blogs:  go to http://gmodestmedblogs.blogspot.com/ to see the blogs in reverse chronological order

  -- click on 3 parallel lines top left, if you want to see blogs by category, then click on "labels" and choose a category​

  -- or you can just click on the magnifying glass on top right, then type in a name in the search box and get all the blogs with that name in them

if you would the article, please email me.

please feel free to circulate this to others. also, if you send me their emails (gmodest@bidmc.harvard.edu), i can add them to the list