ACC guidelines overestimate CAD risk

There have been many concerns about the 2013 Am Heart Assn cardiovasc risk calculator tied to their new cholesterol management guidelines. Several investigators have found huge variations between predicted and actual cardiovascular events (see commentary below). A new article updated the risk calculator based on newer cohort data and revised the statistical methodology to produce a much more accurate risk calculator (see cad risk overestimate 2013 guidelines AIM2018 in dropbox, or doi:10.7326/M17-3011).

Details:

-- The 2 strategies tried were:

    -- updating the database on which the risk calculators were based. The concept here is that several of the studies including the Framingham Heart Study original cohort began in 1948, and it seems appropriate to look at newer cohort data as a basis for the risk calculator (lots has changed since these earlier studies....)

    ​-- different statistical methods: the concern here was that the limited data for African-American patients might lead to “overfitting”,  erroneous estimates for subpopulations with fewer data. They noted, as many others before them, that the risk calculator was frequently extraordinarily inaccurate in African-Americans. There are statistical approaches which can minimize this problem.

-- model 1: they compared the results of the 2013 risk calculator used by the American Heart Association with the risk calculator from updated cohort data
-- model 2: they use the newer data as in model 1, and they created new equations using revised derivation methods to address the overfitting problem as well as problems in how the Cox proportional hazards model was used in the original 2013 AHA risk calculator

Results:
-- the original risk calculator overestimated risk by an average of 20% across risk groups
    -- the risk estimates for black adults could be more than 80% lower to more than 500% higher than those for white adults with otherwise-identical risk factor values
    -- they estimated that 3.9 million US black adults (33%) would have extreme risk estimates (defined as <70% or >250% those of white adults with otherwise-identical non-extreme risk factor values, results that are felt to be highly implausible)
-- model 1 (using updated cohort data for the equation):
    -- slightly improved statistics among black men but not among white adults or black women
-- model 2 (using updated cohort data as well as statistical fixes):
    -- markedly better improvement for all groups, such that fewer than 1% of eligible black adults had their risk estimates appropriately shifted from either being above to  below a 10-year risk score of 7.5% (or vice versa) in the new model, or had the implausible extreme risk estimates (as defined above)
-- Clinical implications of the above:
    -- using the risk cutpoint of 10-yr cardiac risk >7.5%: this translates overall to 11.8 million US adults being recalculated from >7.5% risk using the original calculator to <7.5% with model 2
    -- using the risk cutpoint of >10%: this translates overall to 11.7 million US adults being rescored from >10% risk using the original calculator to <10% with model 2
    --AND, the number of people in the implausible extreme risk estimate group would change: for many black adults (only) one in 29 would be expected to have an implausibly low or high risk estimate
-- some patients, however, may be incorrectly converted from high to low risk status by the new calculator: for each 13 persons who did not have a CVD event but had a risk > 7.5% who were correctly reclassified as low risk, one person who had a CVD event was erroneously reclassified as low risk (13:1 ratio)
-- using the cutpoint of 5% 10-year risk, that ratio improved to 23:1
-- using the cutpoint of 10% 10-year risk, that ratio was 8:1

Commentary:

--as some of you know (those who have been subjected to my blogs for many years), I have been very critical of the 2013 AHA guidelines, finding that some of the recommendations are based on essentially no data (eg, treating younger people, or using a 30-year risk calculator… though I do agree with their recommendations on these) yet some recommendations dismiss lots of data from many studies that are not completely rigorous but really pretty compelling to me (eg, they do not treat LDL to target, which the Treating to New Targets study for example strongly challenges, as well as some of the newer studies on non-statin lipid-lowering meds).  For my review/critique, see http://gmodestmedblogs.blogspot.com/2013/11/new-aha-guidelines-for-risk.html 

--there were several articles that came out soon after the publication of these 2013 guidelines which showed dramatic differences between the predicted and actual events in several different cohorts:

    -- the first study by Ridker and Cook looked at 3 primary prevention cohorts, finding a 75-150% increase in predicted events over observed ones by using the 2013 AHA risk calculator (see chol AHA guidelines overstates risk lancet2013 in dropbox, or Lancet. 2013;382(9907):1762-1765.  Will attach article. also, http://gmodestmedblogs.blogspot.com/2014/04/aha-lipid-guidelines-again.html does give several hypothetical cases (as in the current Annals study), highlighting the huge variance in statin use recommended in the 2013 guidelines vs the older risk calculators

    --see http://gmodestmedblogs.blogspot.com/2014/10/yet-another-analysis-that-lipid.html for a review of Women’s Health Study finding a very poor correlation between the predicted cardiovascular events by the 2013 risk calculator and the actual clinical events, also by Ridker and Cook

--this current Annals article does seem to have tweaked the risk calculator to make it much more accurate, though it needs to be tested in other prospective cohorts

--and this brings up some general concerns about these risk calculators:

    --the relative value of different cardiac risk factors in predicting cardiovascular events does vary by different populations/ethnicities, so one risk calculator may not fit different populations. A recent example is the American Heart Assn Scientific Statement of May 2018 highlighting that the high incidence of cardiovasc events in South Asians (higher than other Asian groups and non-Hispanic whites) is likely related to insulin resistance/metabolic syndrome, and that pretty much all of the risk calculators "have limitations and underestimate CVD risk in South Asians because they have not been derived from or validated in this higher-risk group" (see DOI: 10.1161/CIR.0000000000000580 ), though the QRISK2 calculator in the UK does add South Asian ethnicity as an additional risk factor. 

    --there are basic concerns about applying community data (from which the risk calculators are derived) to individual patients: the issue here is that an individual patient may have an important constellation of cardiac risk factors not included in the risk calculator (perhaps some combo of PAD, migraine, HIV, CKD, albuminuria, migraine, high fibrinogen......) which may put that person at very high cardiovascular risk, yet their risk score may be low by traditional risk calculators. in fact, when these nontraditional risk factors are added to the Framingham model, for example, on a community-wide basis they add insignificantly to the cardiac risk score. The issue is that adding even an important clinical risk factor may not add to the risk calculator for an individual if the community prevalence of that risk factor is low (ie, it would not affect the results because its contribution would be overwhelmed by the common risk factors). but the person with that risk factor may still be at quite elevated cardiovasc risk

--and, another guideline issue: they are not necessarily consistent from one organization to another. see http://gmodestmedblogs.blogspot.com/2018/01/statins-for-primary-cad-prevention-in.html​ which presents the variability of guidelines for statin use in the elderly from 5 international organizations, with more detail on the NICE guidelines from the UK. 

so, my sense is the following:

--i personally never use the 2013 risk calculator because i think it is so inaccurate.

--i do think that whatever risk calculator is used, that it should be treated as a "guideline" in the true sense of the word: an important input in treating patients that provides some potentially useful guidance for treatment, but the actual treatment chosen needs to be based on the individual patient

--i have mentioned in several past blogs that i try to get a "gestalt" of what to do based on the individual patient.  it is really hard to quantify this gestalt. the traditional risk factors clearly hold a lot of sway. other ones (CKD, albuminuria, glucose intolerance/metabolic syndrome, PAD, aortic sclerosis, migraine, inflammatory diseases including treated HIV, prothrombotic disorders.....) do add into my model, and the more present, the higher their input into my assessment of cardiac risk. And, my bottom line is that cardiovascular disease is so prevalent, and the adverse effects of statins so small, that i err to overtreating. By the way, the most common statin problem, myopathy, can often be treated with vitamin D, see: https://blogs.bmj.com/bmjebmspotlight/2017/05/01/primary-care-corner-with-geoffrey-modest-md-statin-myopathy-and-vitamin-d-deficiency/​

--but, that all being said, one of the major issues with risk calculators is that they tend to push statins in a knee-jerk manner. i think that we clinicians should really focus on lifestyle changes with the patient (which helps more than just cardiovascular outcomes, and should be done whether or not a statin is used).  and there are some studies suggesting that some patients revert to unhealthy lifestyles when put on statins ("my cholesterol is so great, i don't need to eat all of those green things and do exercise").  

    -- cardiovascular fitness, for example, is a profound predictor of cardiovascular disease, with suggestions that improvement may dramatically reduce risk, see https://blogs.bmj.com/bmjebmspotlight/2017/01/19/primary-care-corner-with-geoffrey-modest-md-cardiovascular-fitness-a-new-vital-sign/​ , reviewing the AHA scientific statement

    --see http://gmodestmedblogs.blogspot.com/2018/06/mediterranean-diet-with-olive-oil-or.html on the PREDIMED trial posted yesterday, which argues that Mediterranean diet may well be as effective as statins in both primary and secondary heart disease prevention

So I do try to engage the patient along the healthy lifestyle plan: working on a patient-specific plan to optimize diet, exercise, healthy weight, not smoking, decreasing social isolation/stress….., but I also have a lowish threshold to prescribing statins esp in those at quite high cardiovasc risk. That threshold does change on an individual basis, depending on the likelihood of major lifestyle changes, and I do closely follow those patients working on lifestyle changes to see how they are doing and seeing what more they can do….

geoff

If you would like to be on the regular email list for upcoming blogs, please contact me at gmodest@uphams.org

to get access to blogs since 8/15/17:

1. go to http://gmodestmedblogs.blogspot.com/ to see them in reverse chronological order

2. click on 3 parallel lines top left, if you want to see blogs by category, then click on "labels" and choose a category​

3. or you can just type in a name in the search box and get all the blogs with that name in them

to access older blogs from the BMJ website, from October 2013 until 8/15/17: go to http://blogs.bmj.com/bmjebmspotlight/category/archive/ 

please feel free to circulate this to others. also, if you send me their emails, i can add them to the list​