PREVENT: a new cardiac risk factor calculator

 

There has been a concerted effort to improve the cardiovascular risk calculator by the American Heart Association. Their new calculator, called PREVENT (Predicting Risk of CVD EVENTS) was just published (see cardiac risk factor PREVENT circ2024 in dropbox, or DOI: 10.1161/CIRCULATIONAHA.123.067626):

 

Details:

-- derivation cohort (from which the calculator was derived): 3,281,919 individuals with individual-level participant data from 25 data sets, between 1992 and 2017

-- primary outcome: CVD (cardiovascular disease), which included clinical ASCVD (atherosclerotic cardiovascular disease) plus heart failure

-- assessment included traditional risk factors (smoking, systolic blood pressure, cholesterol, antihypertensive or statin meds taken, and diabetes) and estimated GFR (eGFR)

    -- optional predictors assessed included urine albumin-to-creatinine ratio (UACR), hemoglobin A1c, and a “social deprivation index” (SDI, a measure of the individuals’ neighborhoods, which integrated information on seven area-level characteristics including percentage living in poverty, percentage with <12 years education, percentage of single-parent households, percentage living in rented housing units, percentage living in an overcrowded housing unit, percentage of households without a car, and percentage of unemployed adults <65yo; the SDI they used was estimated by inputting the ZIP Code

-- the models were sex-specific, race-free, developed for the age scale of 30-79yo, and adjusted for competing risk of non-CVD death

    -- to view the actual PREDICT calculator, see https://professional.heart.org/en/guidelines-and-statements/prevent-calculator  . This calculator does separate the main items (age, total cholesterol, HDL cholesterol, systolic blood pressure, BMI, eGFR, diabetes, current smoker, antihypertensive medication, lipid lowering medication) from the 3 predictors that are optional to include (UACR, hemoglobin A1c, ZIP Code), mostly because this information was not often available in the databases they used in the validation studies

 

-- external validation cohort (the retrospective analysis of outcomes and published studies using PREVENT versus the older ASCVD risk calculator): 3,330,085 participants from 21 additional data sets.

    -- statistical measurement used is the C-statistic, a measure of the area under the receiver operating characteristic (ROC) curve, which reflects how well a model discriminates between patients who experience an outcome and those who do not; the C-statistic is a single number, where a value of 1.0 reflects perfect concordance, and a value of 0.5 represents a useless prediction (like flipping a coin): see https://www.mcw.edu/-/media/MCW/Departments/Biostatistics/vo19no4.pdf for a brief description of its uses and limitations, as well as https://pubmed.ncbi.nlm.nih.gov/17309939/ 

    -- an assessment of calibration was done, which was calculated by plotting deciles of the predicted vs observed risk of CVD (Iie, derivation vs validation cohorts) and assessing the slope of this relationship, where an optimal calibration is 1.0. A  value <1.0 indicates lower observed than predicted risk (ie overprediction) and if >1.0 an underprediction

 

-- overall mean age 53, 56% women, 78% white/10% Black/6% Latinx

-- mean systolic blood pressure 125 mmHg, total cholesterol 193 mg/dL/non-HDL 135 mg/dL, HDL 54 mg/dL

-- BMI 29, current smoking 6%, diabetes 12%, antihypertension meds 25%, statin treatment 17%, eGFR by creatinine 91

-- UACR 8 mg/g, A1c 7.5% if had diabetes/5.8% if not

-- SDI (social deprivation index): decile 3-4

 

-- main outcomes: the discrimination and calibration of the PREVENT risk predictor as compared to the older ASCVD model

-- mean follow-up 4.8 years

 

Results:

-- total incident CVD events: 211,515  

    -- derivation group: 53,258 in men, 53,403 in women

    -- validation group: 54,365 in men, 50,489 in women

-- ASCVD events:

    -- derivation group: 31,812 in men, 34,691 in women

    -- validation group: 33,969 in men, 33,933 in women

-- Heart failure events:

    -- derivation group: 30,957 in men, 28,393 in women

    -- validation group: 30,287 in men, 25,679 in women

-- Deaths:

    -- derivation group: 84,289 in men, 80,897 in women

    -- validation group: 82,555 in men, 76,783 in women

 

-- meta-analyzed sex-specific adjusted hazard ratios (aHR) of traditional and novel cardiovascular risk predictors for cardiac disease in the derivation model:

    -- the highest correlation value was between eGFR (creatinine-based) and heart failure: aHR 1.96 (1.86-2.07) for females with eGFR <60 (ie, 96% increased risk) and 1.83 (1.69-11.97) for males (83% increase)

    -- other notables: diabetes (87% increase in female and 75% in males), current smoking (75% increase in females and 58% in males); the rest were <40% increase

-- the median C-statistic in the validation cohort was 0.794 (0.763-0.809) for females and 0.757 (0.727- 0.778) for males

    -- similar estimates were found for both atherosclerotic CVD and heart failure models

    -- the improvement in discrimination was small but statistically significant when adding the urine-to-albumin to creatinine ratio, hemoglobin A1c, and the social deprivation index (improvement of 0.004 for females and 0.005 for males, though there was a significant improvement when the UACR was added to the base model inn people with UACR greater than 300 mg/g, improving by 1.39 versus 1.05 in those with lower UACRs. For the PREVENT model, the first value below is for their base model, which does not include the optional inputs; this base model was dramatically better than the old calculator's results:

    -- meta-analyzed discrimination and calibration statistics of model performance for prediction of total cardiovascular disease as well as ASCVD and heart failure in validation cohorts, using the base model:

        -- C-statistic in base PREVENT model: total CVD 0.789 (0.746-0.802) in females, 0.747 (0.727-0.778) in males; ASCVD 0.774 (0.743-0.788) females, 0.736 (0.710-0.755) males; heart failure 0.830 (0.816-0.850) female and 0.809 (0.777-0.827) in males

            -- calibration slope all pretty close to 1.0, though males with heart failure dipped to 0.89

    -- meta-analyzed discrimination and calibration statistics of model performance for prediction of total cardiovascular disease as well as ASCVD and heart failure in validation cohorts, adding only the UACR to the base model:

        -- C-statistic in base PREVENT model enhanced with kidney-specific risk from UACR: total CVD 0.796 (0.766-0.812) in females, 0.759 (0.735-0.780) in males; ASCVD 0.776 (0.746-0.790) females, 0.739 (0.715-0.758) males; heart failure 0.833 (0.820-0.851) female and 0.815 (0.786-0.830) in males [ie, there is a very small benefit by adding the UACR risk]

           -- calibration slope all pretty close to 1.0, though males with heart failure dipped to 0.89 [ie, no significant change by adding the kidney risk]

    -- meta-analyzed discrimination and calibration statistics of model performance for prediction of total cardiovascular disease as well as ASCVD and heart failure in validation cohorts, adding only the hemoglobin A1c:

        -- C-statistic in base PREVENT model enhanced with metabolic risk by adding A1c: total CVD 0.799 (0.771-0.815) in females, 0.759 (0.738-0.780) in males; ASCVD 0.787 (0.750-0.792) females, 0.740 (0.719-0.760) males; heart failure 0.837 (0.818-0.853) female and 0.818 (0.790-0.835) in males [ie, there is a very small  benefit by adding the A1c risk]

            -- calibration slope all pretty close to 1.0, though males with heart failure dipped to 0.88 [ie, no significant change by adding the A1c risk]

    -- meta-analyzed discrimination and calibration statistics of model performance for prediction of total cardiovascular disease as well as ASCVD and heart failure in validation cohorts, adding only SDI:

        -- C-statistic in base PREVENT model enhanced with metabolic risk by adding SDI: total CVD 0.810 (0.788-0.817) in females, 0.774 (0.757-0.789) in males; ASCVD 0.796 (0.761-0.800) in females, 0.753 (0.737-0.774) males; heart failure 0.836 (0.818-0.853) in female and 0.824 (0.793-0.837) in males [ie, there is a very small but a bit more benefit by adding the SDI risk]

            -- calibration slope all pretty close to 1.0, though males with heart failure dipped to 0.84 [ie, no significant change by adding the A1c risk]

    -- meta-analyzed discrimination and calibration statistics of model performance for prediction of total cardiovascular disease as well as ASCVD and heart failure in validation cohorts adding all 3 of the optional add-ons:

        -- C-statistic in base PREVENT model enhanced with all novel predictors: total CVD 0.813 (0.794-0.820) in females, 0.776 (0.762-0.793) in males; ASCVD 0.799 (0.767-0.804) females, 0.755 (0.742-0.776) males; heart failure 0.837 (0.818-0.853) female and 0.818 (0.790-0.835) in males [ie, there is a very small  benefit by adding the 3 optional add-ons]

            -- calibration slope all pretty close to 1.0, though males with heart failure dipped to 0.81 [ie, no significant change by adding all novel predictors]

 

-- estimates of the 10- and 30-year CVD risk:

    -- "the predicted risk for a given age and combination of optimal to suboptimal risk factors varied substantially with a higher estimate with older ages and a dose-dependent relationship with a greater number of elevated risk factor levels"

 

Commentary:

-- routinely including ASCVD (atherosclerotic cardiovascular disease) risk assessment for patients has been suggested by many medical societies for many years and has been largely incorporated into clinical care

-- the current one being used most frequently is the 2013 AHA/ACC one, which included sex- and race- stratified models in White and African-American adults aged 40-79, presence of diabetes, smoking, total and HDL cholesterol, systolic blood pressure and if there was treatment for hypertension (though that 2013 calculator was updated in 2018 to include diastolic blood pressure).  The PREVENT one differs by removing race, increasing the age range to 30-79, and adding questions about treatment with statins, BMI, creatinine-based eGFR, along with the optional predictors of UACR, A1c level and ZIP code.  Also, they separately assessed heart failure as a cardiac outcome given both the large increases in heart failure and also our morphing to a different concept of cardiovascular risk factors, from one highlighting the long-known traditional ones for atherosclerosis (smoking, hypertension, dyslipidemia, diabetes) to a CKM (cardiovascular-kidney-metabolic) model.  Some concerns about the development of this new risk model:

    -- they are largely abandoning race-based models appropriately, given that race is a social construct and not a genetic determinant. it should be noted that the prior race-based model was limited to "White, African-American, and Other" as categories, though other races or ethnicities, so it did not even reflect the intended diversity of the population

    -- they do not include several other documented risk factors (stress, depression, air pollution, presence of microplastics, migraine, rheumatoid arthritis/psoriasis/lupus, chronic infections (including chronic hepatitis, HIV even if controlled, etc). see https://gmodestmedblogs.blogspot.com/2023/10/update-ascvd-risk-factor-critique.html, which also raises concerns about the lack of important details to appropriately incorporate even the traditional risk factors

    -- this study also found that eGFR <60 actually had the highest relative risk for heart failure. there is pretty clearly a relationship between CKD and heart failure (hence the CKM model), with a recent study suggesting benefit of measuring regular BNP tests in all with CKD every 3 months: https://gmodestmedblogs.blogspot.com/2023/12/routine-bnp-assessment-helpful-for.html

        -- in this context, it would be useful to have a breakdown of cardiac risk by stages of CKD and not just the binary one of eGFR <60 vs >60

        -- and, using cystatin-based eGFR is better than creatinine-based eGFR for predicting clinical outcomes (though the tool incorporating both cystatin- as well as creatinine-based measures may more truly represent the measured eGFR): https://gmodestmedblogs.blogspot.com/2023/12/cystatin-c-better-predictor-of-bad.html 

 

-- and this brings up my other concerns about the PREVENT risk prediction tool:

    -- it only includes systolic blood pressure, and diastolic blood pressure does confer cardiovascular risk

    -- some of these are binary components (eg, cigarette smoking), and we know that passive smoking is important, the number of years smoking at high versus low quantity smoked is important, how recently a person stopped smoking, ...)

    -- some rely on surrogate markers (for a much more exhaustive assessment of surrogate markers and their limitations, see https://gmodestmedblogs.blogspot.com/2024/06/using-surrogate-markers-for-disease-are.html ), but in brief:

        -- diabetes: A1c is a surrogate marker, the cutpoint of 6.5% is not based on the most frequent and dangerous clinical outcome of cardiovascular disease, but is based on when retinopathy just starts to increase in incidence at an A1c of 6.5%; the meds used are really important (GLP-1s and SGLT-2s are cardio/renal protective, insulin/sulfonylureas are cardiotoxic in several studies). Even the cutpoint of 5.7% for glucose intolerance is chosen for unclear reasons (several epidemiological studies have found that men with A1c levels even below 5.5% have a significant increase in cardiovascular events). 

        --lipid makers: both LDL and HDL are not great markers for subsequent cardiac events: several studies have found that apolipoprotein B is a better predictor than LDL of cardiac harm and apolipoprotein A is better than HDL for cardiac protection (both LDL and HDL include different subtypes that migrate electrophoretically in the same broad LDL and HDL classes; small/dense LDL is 3x as cardiotoxic as the bigger one, HDL with apolipoprotein C3 instead of A is actually quite cardiotoxic....)

        -- BMI: less predictive of cardiac events than visceral abdominal obesity

        -- using ZIP codes as a surrogate for the multifactorial issues about social deprivation is also quite fraught: 

            -- especially in more populous urban areas, there is a lot of variability of income, wealth, housing, education level, etc within the same ZIP code

            -- the social deprivation index is a conglomeration of many different factors. as with many of the composite measures, there is equal value applied for components that include items with unequal actual values: should having a car be given the same value as living in poverty??

        -- and this PREVENT risk calculator does not include many other causes of chronic inflammation which tend to be associated with cardiovascular disease: rheumatoid arthritis/psoriasis/lupus, HIV even if controlled, other chronic infections, air pollution, stress, depression, migraine, microplastic exposure, or even other atherosclerotic diseases such as PAD:  in brief, all of these confer significant cardiovascular risk, though these may not be common enough or routinely assessed in the large observational studies used to create the ASCVD risk calculators, to be applicable to the specific patient in front of us. (eg, not lots of people with HIV in the Framingham cohort to find higher cardiovascular rate in the overall population; similarly would psoriasis or rheumatoid arthritis bubble up to statistical significance in large populations where most people have hypertension, obesity, dyslipidemia, etc??)

-- though the C-statistic is pretty high, that again is based on community data and not the specifics of an individual (how does one integrate the significant cardiovascular risk factors of depression, stress, migraine, chronic hepatitis, HIV, exposure to high levels of air pollution or microplastics, etc etc into the clinical decision on how aggressively to treat their dyslipidemia or other more treatable risk factors to lower the risk? 

    – similarly, the calibration slope overall is quite high. but we are comparing derivation and calibration cohorts that both are pretty healthy populations that may well not represent the individual patients we see (and these are the ones where we need to make our clinical therapeutic interventions)

 

-- Another study was done recently to compare the new PREVENT calculator to the 2013 ASCVD risk calculator, using the 2019 AHA/ACC guidelines on primary cardiovascular disease prevention, in 3785 US adults (mean age 56, 53% women; the sample was weighted to mimic the current US population) without known prior clinical ASCVD. Data was from the NHANES survey from 2017-2020, see ASCVD vs PREVENT models JAMAintmed2024 in dropbox, or doi:10.1001/jamainternmed.2024.1302. 

    --mean estimated 10-yr risk by old tool was 8.0%

    --mean estimated 10-yr risk by PREVENT was 4.3%

        -- for individuals aged 70-75yo, the difference in meeting the accepted primary care prevention cutpoint to prescribe statins would decrease from 22.8% to 10.2%

    --overall, this decrease using the PREVENT calculator would translate to a reduction of the number of adults meeting the criteria for primary prevention statin therapy from 45.4 million to 28.3 million

    --of course, this is assuming that the guidelines for when to start statin therapy for primary prevention are appropriate, though:

        -- cardiovascular disease is still the leading cause of death in the elderly

        -- 25% of elderly people without a history of cardiovascular disease still die from it

        -- and we clearly know that atherosclerosis is a progressive disease beginning in teens and typically without symptoms for many decades in the vast majority of people

        -- statins lower relative risk for cardiac events about 30% in both primary and secondary prevention. but about 20-30% of people having an acute MI die (most prior to reaching the hospital): an important point in favor of vigorous primary prevention

        -- the absolute risk of a cardiac event increases dramatically with age

        -- observational studies (eg, a VA one) found statin benefit in those >75yo (https://gmodestmedblogs.blogspot.com/2020/07/elderly-statins-help.html ); this was also found in subgroup analysis of 26 RCTs (https://www.thelancet.com/article/S0140-6736(10)61350-5/fulltext )

        -- so, one could argue that our threshold for statin use should be much lower than the current one in order to prevent the mortality and morbidity of atherosclerotic disease with aging

            -- that being said, i have been much more aggressive than the AHA/ACC guidelines for many years, even using high intensity statins in patients well over 90yo. the statins have been very well-tolerated. and very few of have had clinical cardiovascular disease. this is anecdotal, but i do see lots of people >90yo. i should add, and emphasize, that i do work hard with patients to help them improve their lifestyle (diet, exercise, weight loss when indicated, smoking/alcohol cessation, etc) to improve their cardiovascular risk profile. though many still need statins

            -- i also realize that there are not any RCTs justifying treating patients >75yo (hence the guideline age limit), but the large numbers of those >75yo who die from cardiac disease but have not had a prior event (ie, primary prevention), along with the known pathophysiology of atherosclerosis extending beyond 75yo, is to me pretty compelling that it is appropriate to treat people more aggressively with nonpharmacologic and medical therapy as needed

 

Limitations:

-- While the C-statistic can be a useful tool for describing the discrimination ability of a predictive model, it is important to keep in mind its limitations (see the Details section above which describes the C-statistic). One concern is that the scale of the C-statistic is condensed to a single number so that even important predictive variables with a substantial odds ratio of cardiac disease in individuals may have an insignificant impact on the population-based C-statistic. The calibration statistic is also important in assessing the importance of a risk factor, and calibration is assessed by the specifics of patients included in the derivation and validation cohorts (which may also be quite limited, as noted above).

    -- the study-derived C-statistics above were somewhat under 0.8, and there is argument in the literature that between 0.6 and 0.8 indicates moderate diagnostic accuracy and between 0.81-0.9 good diagnostic accuracy. so, this model, though a step forward, still needs revisions

    -- overall, though this was a very large group of individuals and with similar baseline characteristics in the derivation and the validation groups, they were quite healthy: excellent systolic blood pressure at 125 mmHg, 11% with diabetes, 6% smokers (though 4% in validation group), 25% on antihypertension meds and 16% on statins, and SDI in the 3^rd and 4th deciles

        -- and the median age was 53 (so not sure how these results would apply to a 30yo, the lower limit for the PREVENT calculator (there is no breakdown of PREVENT by age in the article or supplementary materials, other than the comment that there were higher cardiac risk estimates with older age. since atherosclerosis starts in youth, should there be a lower cutpoint to more aggressive therapy in younger people than in older ones???

-- several of the measured components are binary, as mentioned above, which does not allow for important gradations, such as patients who smoke a couple cigarettes a day versus to two packs a day do not have the same cardiovascular risk. and those who quit smoking recently or have passive smoke exposure are considered in the same group as nonsmokers, though they are also at high cardiovascular risk (in fact, some studies find smoking to have the highest risk of the traditional risk factors). Even some that are not binary in the analysis, such as hypertension, are included at relatively broad intervals of 20 mmHg. people with elevated UACR are also at increased cardiovasc risk, with some studies finding even lower numbers (eg even below 20) are associated with increased cardiovascular risk. their cutpoint of 300 is pretty high, and there were few patients in the groups assessed in the  much lower numbers (i could not find that specific information in either the article or supplement, just a comment by them that this was the case)

-- the median followup was only 4.8 years in the derivation sample. i am concerned about their 30-year risk prediction for several reasons:

   -- the information from the validation studies is very old. the baseline characteristics of the patients in these long-term observational studies is fraught:

        -- many of the demographics have changed: more obesity now. more diabetes. changes in air quality and other environmental issues. differences in social and work stressors. smoking is less now

        -- many of the treatments/meds are different: meds for diabetes and hypertension are better now. goals of treatment are more appropriate now.

        -- long-term studies often do not have sufficient data on lifestyle issues, meds taken etc, but rely on baseline evaluation or quite intermittent evaluations, and they may well miss important changes during the course of these long-term studies 

        -- per their own comments (quoted above), the predicted risk "varied substantially", especially in the 30-year projections

-- it is also curious that they did not include any assessment of renal outcomes in this analysis, since they highlight the CKM (cardiovascular-kidney-metabolic) model, and so many of the cardiac risk factors affect renal function

 

so,

-- it is certainly important to refine our predictive models for cardiac disease to improve their accuracy, as is done by the PREVENT cardiac risk calculator.

-- it is certainly important to consider a 30-year predictive time-frame, especially for younger people, as is done by the PREVENT cardiac risk calculator.

-- given that the atherosclerosis process begins at a young age, it is also important to be able to predict accurately the higher cardiac risk to a younger age in order to consider non-pharmacologic and pharmacologic interventions, and PREVENT goes back to age 30

-- but, though PREVENT is clearly better than prior calculators, there are some inherent difficulties in developing a much more robust calculator in the future:

    -- we need more robust long-term observational studies from which we can derive more useful data: the current ones do not have some very important features, such as granular data on pretty much all of the potential risk factors (eg, for smokers, length of time smoking, changes in smoking over time with perhaps yearly updates, exposure to passive smoking, etc. Similarly for diet, exercise, socioeconomic parameters, obesity, etc). Would be great to also have data on meds taken (are people with diabetes on cardioprotective meds and for how long, what have been changes in meds taken over time, what are differences in A1c levels over time,…)

    -- we need age-appropriate thresholds for action: young people even at high cardiovascular risk are not going to have a high risk score. We need clear alerts as to what is predictive of problems in the more distant future in order to act sooner to stem the progression of atherosclerosis

    -- we need databases with broader, sicker and more diverse social populations in order to assure generalizability to the population as a whole

    -- though we do need to rely on surrogate markers to predict outcomes, we need to understand their limits openly and attempt to use the best of them (eg waist circumference is better than BMI as a reflection of obesity’s risk to atherosclerosis; cystatin-based eGFR assessments)

    -- we need to include cardiotoxic components that apply to individuals but may not be evident in large samples of people, either because they are relatively uncommon or because they are not measured: perceived stress, rheumatologic diseases, chronic infections, migraine, depression, air pollution, etc (eg, pretty much anything associated with chronic systemic inflammation)

    -- and, the sooner we develop these databases, the better, since it will take many years of data collection to get the answers we need.

-- overall, the PREVENT calculator is a great step forward, but it is clearly limited by lack of broader and more granular data input into the model

geoff

-----------------------------------

If you would like to be on the regular email list for upcoming blogs, please contact me at gmodest@bidmc.harvard.edu

  

to get access to all of the blogs:  go to http://gmodestmedblogs.blogspot.com/ to see the blogs in reverse chronological order

  -- click on 3 parallel lines top left, if you want to see blogs by category, then click on "labels" and choose a category​

  -- or you can just click on the magnifying glass on top right, then type in a name in the search box and get all the blogs with that name in them

 

if you would the article, please email me.

 

please feel free to circulate this to others. also, if you send me their emails (gmodest@bidmc.harvard.edu), i can add them to the list