Cochrane review critique: don't get rid of your masks yet

 A  recent Cochrane review assessed  mask effectiveness in preventing the spread of respiratory viruses, though with what seems to be inappropriate extension to SAR-SoV-2  (https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD006207.pub6/full: You have the options of viewing the full article (326 pages), the standard article (a measly, indulge in the pun, 305 pages which does not include data and analyses), or the summary (only 5 pages).  This review hit the national press right away (eg see https://www.nytimes.com/2023/02/21/opinion/do-mask-mandates-work.html?smid=nytcore-ios-share&referringSource=articleShare , with perhaps the pretty distorted title in the New York Times editorial by Bret Stephens: “The Mask Mandates Did Nothing”

 

Details: 

-- this Cochrane review included 78 RCTs, 6 of which were conducted during the Covid pandemic: 2 from Mexico, one each from Denmark, Bangladesh, England, and Norway. There were also 4 ongoing studies, one of which is completed but unreported, evaluating mask use during Covid. [ie: very few studies on SARS-CoV-2, most were conducted during pre-Covid non-epidemic influenza times]. 

 

 Results: 

organized by their 3 categories (many of my comments included in brackets):

 

medical/surgical masks versus no masks:

 -- 12 trials, 2 of healthcare workers and 10 in the community:

    -- community studies: 9 trials, 276,917 participants, risk ratio 0.95 (0.84-1.09), moderate certainty evidence of no statistically significant benefit

    --  for laboratory confirmed influenza/SARS-CoV-2: 6 trials, 13,919 participants, RR 1.01 (0.72-1.46), not statistically significant

    -- harms were rarely measured or well-reported 

    -- [to my review of their large document, there were only 2 studies that assessed mask-using during Covid; see analysis of these studies below]

 

N95/P2 respirators versus medical/surgical masks:

-- 4 studies in healthcare settings, 1 in household setting

    --  overall pooled results: 3 trials, 7779 participants, RR 0.70 (0.45-1.10), very low certainty evidence

        -- for influenza-like illness (ILI): 3 trials, 7779 participants, RR 0.82 (0.66-1.03), low certainty of evidence (because of imprecision and heterogeneity for these subjective outcomes)

        -- for laboratory-confirmed influenza infection, 5 trials, 8407 participants, RR 1.10 (0.90-1.34), moderate certainty evidence of no benefit

        -- healthcare worker studies: no difference in overall findings

-- a recent RCT was just published of 1009 healthcare workers in 4 countries who were providing direct care to Covid 19 patients, finding that medical/surgical masks were non-inferior to N95 respirators

-- harms were poorly measured or reported

 

Hand hygiene compared to control

-- 19 trials included in a meta-analysis, with settings including schools, childcare centers and homes

    -- acute respiratory infections: 9 trials with 52,105 participants: 14% relative reduction, RR 0.86 (0.81-0.90), statistically significant, moderate-certainty evidence

        -- absolute benefit: reduction in risk from 380 events/1000 people to 327/1000 people (308-342)

    -- with more strictly defined ILI: 11 trials with 34,503 participants, RR 0.94 (0.81-1.09), not statistically significant

    -- laboratory-confirmed influenza: 9  trials with 8332 participants, RR 0.91 (0.63-1.30), not statistically significant

-- few trials measured or reported harms

 -- [One issue here is that they are combining studies in very different settings, and it may well be that those done in schools and perhaps in child care settings may well have required several mitigation strategies more intensively than at home, and combining the results of these sites in their review may not be appropriate]

 

-- there were no RCTs on gowns and gloves, face shields, or screening at entry ports

  

Authors’ conclusions:

    -- difficult to draw firm conclusions since there was high risk of bias in the trials, variation and outcome measurement, and relatively low inherence with the interventions during studies

    -- “There were additional RCTs during the pandemic related to physical interventions but a relative paucity given the importance of the question of masking and its relative effectiveness and the concomitant measures of mask adherence which would be highly relevant to the measurement of effectiveness, especially in the elderly and in young children” [see below. They did exclude several studies with positive results for masking]

 

Commentary:

-- One big issue with the above analysis is extending the conclusions of their review to include the current Covid situation: the infectivity of the newer variants is significantly more than the ones involved in the cited studies.

    -- R0 (R-naught), is the accepted measure of the contagiousness of pathogens, where an R0 >1 suggests that the pathogen will be infecting a greater number of people than the index person: ie, if one person spreads it to 2 others, that would be an R0 of 2,, making the pathogen’s effect on the population increasing, sometimes to pandemic proportions. The viral infection considered to have the highest R0 is measles, with an R0 of about 18

    -- the R0 is a an average measure, not fixed. For example, as many people in the community become less susceptible (eg through vaccination, or completed infection), the R0 would be expected to decrease. Several studies report the “basic reproduction rate” as the anticipated contagion rate if no one had immunity from infection, and the “effective reproduction rate” is what would happen in the real world if there was some ambient immunity from vaccination or past infection

    -- the R0 for influenza virus is in typically in the 1-2 range

    -- the R0 for the ancestral SARS-CoV-2 strain (the apparent one in several of the Covid studies, which were all done from data earlier in the pandemic) was 2.8; the Delta strain (which may have played some role in the studies) was 5.1 (see https://academic.oup.com/jtm/article/28/7/taab124/6346388 ), and the initial omicron variant had an average basic reproduction rate that was 9.5 (range 5.5-24, median 10; the R0 was 24 in South Africa) and the average effective reproduction rate was 3.4 (0.88-9.4, median 2.8); these numbers for Omicron were 3.8 and 2.5 times higher than the transmissibility of Delta (see https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8992231/#:~:text=The%20Omicron%20variant%20has%20an%20average%20basic%20reproduction%20number%20of,IQR%3A%202.03%2C%203.85).   the Delta variant had about twice the transmissibility of the ancestral strain (https://pubmed.ncbi.nlm.nih.gov/34369565/ )  

    -- and, and, and the new variants (eg XBB.1.5 is the most transmissible one now….) 

--so, making statements that masking would not help much now, as they are based on older SARS-CoV-2 variants: ie drawing conclusions about covid would not be scientifically valid!!! 

 

-- the next big issue with the Cochrane review is that they misrepresented the  Covid studies they reviewed on masking in the community. 

 

1.Abaluck’s study, https://www.science.org/doi/10.1126/science.abi9069 : a 2021 Bangladesh study of 342,183 adults (mostly men) in 600 villages given free masks for 2 months and compared to villages without the masks (control villages) to see if mask-wearing decreased symptomatic SARS-CoV-2 infections. They also assessed differences between cloth vs surgical masks. To stress the importance of masking, there was role-modeling by community leaders and in-person reminders.

-- outcomes: proper mask-wearing, physical distancing (at least 1 arm’s length from nearest adult), social distancing (total number of adults observed in public areas through direct observation at mosques, markets, main entrance roads to villages, and tea stalls), and symptoms of covid. Followup in 5 and 9 weeks. Blood samples of symptomatic patients were checked for SARS-CoV-2 IgG antibodies [but serum antibodies may reflect old infection from before the study and not a new infection. And we do not know the prevalence of covid infections in those areas]

--results:  proper mask-wearing increased from 13.3% in control villages to 42.3% in treatment villages throughout intervention [and, perhaps one needs a higher mask-wearing rate to really affect outcome change; or maybe there is a much stronger effect of masking if there were more physical distancing beyond one arm’s length??]

--physical distance increased in 24.1% in control villages to 29.2% in treatment villages [not so much change]

--no change in social distancing [they defined this as the total number of adults observed in public areas: ??how many people, in how large an area?? What was the population density in those areas?]

-- after 5 months: intervention on mask-wearing waned but remained 10% higher in the intervention villages

-- covid-like symptoms: 7.63% (12,784 people) in intervention villages and 8.60% in control ones (13,287 people), 9.5% decrease, adjusted prevalence ratio 0.91 (0.82-1.00), statistically significant; and by using the WHO covid symptom metric, there was a 11.5% decrease, intervention effect of 0.885 (0.834-0.934)

    -- in people randomized to surgical masks, decrease of 11.1%, adjusted prevalence ratio 0.89 (0.78-1.00),[ ie more than with cloth masks]

    -- in people >60yo: a pretty impressive 35% decrease, adjusted prevalence ratio 0.65 (0.45-0.85) 

-- and, larger reductions in symptoms and symptomatic seropositivity in villages with larger increases in mask use [herd immunity…]

-- there was some benefit of the intervention on physical distancing, none on social distancing

-- so, despite the negative effects published in the Cochrane review, there was quite a significant benefit to mask-wearing overall  (statistically significant  for Covid symptoms and seropositivity) and especially in those >60yo (35% decrease) wearing surgical masks, and in those areas with higher mask usage!! And this was done in 2021, prior to much more contagious variants of SARS-CoV-2!!!! 

  

2.Bundgaard, https://www.acpjournals.org/doi/10.7326/M20-6817 : March 2021 Danish study of 3030 adults spending >3h/d outside the home without masks, randomized to recommendations to wear 3-layer surgical masks vs controls

-- Covid infection in 42 people recommended to wear masks (1.8%) vs 53 controls (2.1%), not statistically significant

-- 46% wore masks as recommended 

-- very low numbers getting covid, likely reflecting the earlier variants involved and their being less communicable 

-- main conclusion: “those wearing surgical masks to supplement other public health measures did not reduce the SARS-CoV-2 infection rate by more than 50% in a community with modest infection rates, some degree of social distancing, and uncommon general mask use” [ie:  this was not a study on masks, and they do not disaggregate masks from other mitigation strategies.]  

  

And, one study on health care workers: 

3.Loeb, https://www.acpjournals.org/doi/10.7326/M22-1966 : 1009 health care workers in Canada, Israel, Pakistan, and Egypt from May 2020 til March 2022, comparing medical masks vs fit-tested N95s for 10 weeks. Overall no statistically significant difference, by comparing medical vs N95 with PCR-confirmed covid. but, by countries:

    -- Canada: 8 of 131 people (6.11%) vs 3 of 135 (2.22%), almost 3-fold difference, HR 2.83 (0.75-10.72) 

    -- Israel  and Pakistan: too few people in the study to have statistically reliable numbers

    -- Egypt (during omicron): 35 of 257 people (13.6%) vs 28 of 261(14.6%), nonsignificant (had omicron exposure) 

    -- 11% had adverse event in medical mask group vs 14% with N95 (discomfort, skin irritation, headaches) 

-- [hard to interpret this study: huge benefit in Canada but not in other areas. Some of the difference attributable to the methodology and timing: Canada’s component was early in the pandemic in acute health care facilities; Israel’s was in long-term care facilities where there already had been large outbreaks); Pakistan and Egypt were late in the pandemic and  the studies were undertaken in very densely populated areas so very likely high incidence of old covid infections. So, harder to make the case that there was no difference between medical masks and N95s]

  

-- That all being said, this review was of studies on limiting the spread of respiratory viruses mostly before Covid, though they make such comments as: "wearing masks in the community probably makes little or no difference to the outcomes of laboratory-confirmed influenza/SARS-CoV-2 compared to not wearing masks", commenting that this was with "moderate-certainty evidence", based on the 2 trials above (which hardly support this claim as being of moderate certainty...). this conclusion seems to be a counterintuitive conclusion:

    -- in terms of influenza and other respiratory viruses, it does seem that these were dramatically decreased during the covid pandemic when people were taking precautions. and, my guess, the major precaution taken by most people was mask-wearing (a visible sign to all) and distancing, and less so for aggressive hand washing (which pretty clearly is important).

    -- and there were several articles that supported the effectiveness of masks in covid: 

        -- see https://bucommunitymed.wpengine.com/covid-the-importance-of-masks-and-ventilation/ which reviews 10 studies on the benefit of mask-wearing

        -- or https://bucommunitymed.wpengine.com/covid-do-masks-work-they-do-in-kansas/ , a CDC report documenting covid incidence in different areas of Kansas, noting decreased covid in counties with mandated mask-wearing vs counties without

        -- and https://bucommunitymed.wpengine.com/covid-transmission-and-distancing/ , which reviewed 10 articles on the distance that respiratory droplets can spread, and that in some studies it exceeded the 6 feet of social distancing recommended, with one study finding droplets up to 26 feet away

            -- this becomes even more of an issue when the transmission rate (R0) is so much higher in the newer SARS-CoV-2 variants

            -- by the way, the best of the covid articles cited in the Cochrane review (Abaluck's study) defined appropriate distancing at an arm's length (and my guess is that few of us have arms reaching 6 feet....).  Perhaps the viral spread is so intense at an arm's length that masks work less well (viral load should decrease rapidly with distance, by the inverse of distance squared if i remember my physics well....). And, the viral load is much higher with omicron vs delta, as well as there being more sustained viral shedding (see https://pubmed.ncbi.nlm.nih.gov/36366518/#:~:text=Viral%20load%20was%20measured%20as,cases%2C%20especially%20in%20the%20nasopharynx. ) 

 

Limitations:

-- there were only 2 articles that included mask usage in Covid spread in the community in the Cochrane review. other studies were excluded (including the ones from the blogs just mentioned). My reading these 2 included articles made it pretty clear that they were not great studies on which to make a broad conclusion that masks do not work in the community. and, as per the above, they only dealt with older variants, mostly pre-delta, and the current variants of SARS-CoV-2 are so much more transmissible....  

-- there was limited adherence in these studies with mask wearing. the differential between the experimental and control groups of actual amount of mask-wearing was not huge, and even if these studies were well-done and accurate, there may need to be a higher level of mask-wearing to protect the wearer and others. not sure what that threshold would be, but it would be undoubtedly higher with more aggressive SARS-CoV-2 variants

-- systematic reviews combining studies from very different areas, having different populations, with different ambient conditions (eg the prevalence of Covid in the community/prevalence of past infections/prevalence of which SARS-CoV-2 variant is around/cultural differences in person-to-person interactions and closeness), using quite different methodologies (some with N95, others with medical masks, others with cloth masks) and definitions (what constitutes physical or social distancing), and having different designs (masks only in some studies vs masks with other mitigation strategies in others, the latter making it difficult to disentangle the actual effects of masks by itself). adding the disparate studies together is a tad fraught....

-- no data at all on the effect on asymptomatic infections, which could spread easily to high-risk people (those who had symptomatic covid might avoid being around higher risk people)

 

So, this review brings up several issues:

 

-- the argument that mask wearing does nothing in the current situation is, I think, not likely to be accurate, and is not demonstrated in this Cochrane review. It was pretty striking to me that in my reading the 2 relevant studies on mask-wearing and Covid, the published results were pretty different from what is in the Cochrane reviews. For example, the Bangladesh study certainly had some flaws as above, but their researchers’ overall conclusion was that masks do help a bit, and they were quite helpful in those >60yo (35% decrease in symptomatic covid). their actual conclusion: "the intervention increased mask usage and reduced symptomatic SARS-CoV-2 infections, demonstrating that promoting community mask-wearing can improve public health".....

  

-- there is an inherent problem releasing such a negative assessment of masks to the press prior to our being able to review them  (eg, I had a patient that day the report was released who said that we did not need to wear masks anymore and asked if he could remove his since masks were shown to do nothing). Not sure how to fix this problem, but it does create some chaos and confusion. And might well lead to more covid cases

-- it is impossible to extend the results of early studies of SARS-CoV-2 to the current situation since the infectivity of the virus early on was much less than with the current variants. And the Cochrane review did not have a clear message: at some points they suggested that their results included SARS-CoV-2 and at others they were more circumspect, acknowledging a data gap

-- the quality of this review does raise the issue that we should be more diligent before accepting the conclusions of reviews by the Cochrane collaborative

 

-- and, one really, really important issue: in the current right-wing political swings in the US and abroad, this "negative study" on masks (and generalized in the pretty awful NYTimes editorial trashing mask wearing and the imposition of  “mask mandates") further reinforces the public undercurrent of not believing in science (and the politicians/news media promoting anti-science), with the potentially awful future implications: not fearing a bird flu outbreak with really high mortality, not getting the array of important vaccinations (eg against measles), dismissing climate change or the importance of biological diversity, etc...

 

 

geoff

----------------------------------- 

If you would like to be on the regular email list for upcoming blogs, please contact me at gmodest@bidmc.harvard.edu 

   

to get access to all of the blogs: go to https://bucommunitymed.wpengine.com/, a website from the Community Medicine section at Boston Medical Center.  This site does have a very searchable and accessible list of my blogs and is the easiest to view blogs and displays more at a time. 

 

please feel free to circulate this to others. also, if you send me their emails, i can add them to the list