guidelines: lacking evidence-based medicine

A recent study updated one 10 years ago, finding that there was no improvement in the level of evidence in cardiology guidelines, with the majority still relying on expert opinion both in Europe and the US (see ebm card guidelines jama2019 in dropbox, or doi:10.1001/jama.2019.1122).

Details:

-- 26 current ACC/AHA guidelines published from 2008-2018 were evaluated

    -- 2930 recommendations, median 121 per guideline

-- 25 current ESC (European Society of Cardiology) guidelines from 2003-2018 were evaluated

    -- 3399 recommendations, median 130 per guideline

-- they evaluated the level of evidence (LOE) for the recommendations:

    -- LOE A: multiple RCTs are single large RCT

    -- LOE B: observational studies or a single RCT

    -- LOE C: expert opinion only

Results:

-- ACC/AHA:

    -- LOE A: 8.5%

    -- LOE B: 50.0%

    -- LOE C: 41.5%

    -- of those recommendations considered Class 1 (there is evidence, general agreement, or both, that the treatment is useful or effective): 1272 recommendations, 180 (14.2%) were classified as LOE A

    -- since 2015, ACC/AHA guidelines with LOE B were subdivided into observational studies vs RCT: only 26.5% were based on RCTs

    -- in terms of trends over time: 5.7% of recommendations in the past 2 years were LOE A (ie, lower than before, which was 9.5%)

-- ESC:

    -- LOE A: 14.2 %

    -- LOE B: 31.0 %

    -- LOE C: 54.8 %

    -- of those recommendations considered Class 1 or Class 3 (same as Class 1, but treatment is not useful or effective and may be harmful): 21.3% were LOE A

    --in terms of trends over time: 17.5% of recommendations in the past 2 years were LOE A (ie, higher than before, which was 12.8%)

-- the proportion of recommendations that were LOE A, comparing current guidelines to prior ones 10 years before:

    -- ACC/AHA: median 9.0% (current) vs 11.7% (prior)

    -- ESC: median 15.1% (current) vs 17.6% (prior)

    -- no meaningful difference between 2008 and 2018; and no meaningful change in the class 1 recommendations

Commentary:

-- pretty striking finding of no change in the quality of recommendations by 2 of the most well-funded and advanced specialty societies in the world, in an era of increased emphasis on evidence-based medicine

-- the large majority of patient care recommendations were based on nonrandomized evidence, even those considered to be “should do”or “should not do”

-- there was substantial variability depending on the recommendations, with some having 0% LOE A and some up to 33%

-- one obvious trend in studies, as reflected in many of my  blogs over the last couple of years, is more use of big data-mining studies. These are useful, pretty easy-to-do studies typically mining data from large databases (many from insurance companies). But, they are not RCTs, from which clear conclusions/causal relationships can be derived. and their patient populations themselves may not be so generalizable (eg, many studies reflect data from private insurance databases)

-- why has there not been an improvement in the rigor behind recommendations? No doubt several factors:

    -- more and more, drug companies support and define the studies being done

        -- they are not very likely to perform studies looking at routine history, physical exam, basic lab tests, nonpharmacologic interventions, or even appropriate comparisons between new and old medications (does it really make sense to compare simvastatin 40mg plus ezetimibe to simvastatin 40mg by itself, instead of to a higher potency statin??, except that the drug company Merck funding the study makes both of these drugs?? see http://gmodestmedblogs.blogspot.com/2015/06/improve-it-trial-ezetimibe.html

        -- and, the focus of their studies is to achieve regulatory approval for their specific drug or other theapy, and not necessarily to provide clear and useful clinical information

-- and, who are these “experts” who make these guidelines and upon whose expert opinion many recommendations are based? No data on this. But, my guess is many of them are researchers, administrators, and academicians. And, I’m not sure many of them are steeped in the actual clinical practice of medicine, and their perspective/expert opinions may reflect these realities. And they may be swayed by which drug company is supporting their specific research

-- as a sideline to the above: though randomized controlled trials are considered the gold standard for evidence-based medicine, there are impressive limitations to the generalizability of results from these RCTs.  They typically exclude patients who are more complex and have more comorbidities, or elderly patients, or pregnant ones, or younger ones, etc. And they do not include a wide array of patients from different ethnic/racial backgrounds. And they often have a very narrow scope of how they define their group of patients (often with limited socioeconomic and psychosocial parameters). And, no matter how much they mathematically adjust for the variations within their patients, there is residual uncertainty on how to apply the results to the specific patient in front of us (who, undoubtedly, varies in many ways from the somewhat reductionist analysis of their average study patient who is 56.8 years old, 53% female, 76% white, 23% has heart failure, 61% on statins, and with no known psychosocial issues that could affect their disease process …….)

so,  i do not mean to be totally nihilistic in pointing out the limitations of guidelines, though it is notable that there has been no progression in the availability of higher quality studies upon which to make the recommendations, at least in cardiology. it's just that guidelines should not be blindly applied to the individual patients being seen, and are only a "guide" on what some "experts" think is the right (or wrong) approach to patients.  these  recommendations are reasonable to consider, but should be assessed critically both in general (ie, should all diabetics >65 yo really be seen by specialists?? as in https://gmodestmedblogs.blogspot.com/2019/04/diabetes-care-in-those-65yo.html), as well as in terms of the specific individual patient in front of us

geoff

If you would like to be on the regular email list for upcoming blogs, please contact me at gmodest@uphams.org

to get access to blogs since 8/15/17:

1. go to http://gmodestmedblogs.blogspot.com/ to see them in reverse chronological order

2. click on 3 parallel lines top left, if you want to see blogs by category, then click on "labels" and choose a category

3. or you can just type in a name in the search box and get all the blogs with that name in them

to access older blogs from the BMJ website, from October 2013 until 8/15/17: go to http://blogs.bmj.com/bmjebmspotlight/category/archive/ 

please feel free to circulate this to others. also, if you send me their emails, i can add them to the list