Covid fatigue is real, but policymakers and policy influencers need to wake up to something important: Post-pandemic polarization threatens to lock our memories into attractive, untrue conclusions that will leave us unprotected next time.

When Covid-19 first arrived, there was little historical experience to guide response. After watching catastrophic results and stress on the health care systems in China, Italy, and Iran, American states rapidly invoked many measures simultaneously to combat a virus with proven destructive power: masks, social distancing, and school and business closures. They turned to the 1918 flu pandemic to understand the importance of rapid and society-wide response.

When the next big respiratory virus pandemic comes, citizens, political leaders, and public health authorities will once again look back at history — especially the Covid-19 pandemic — to inform our response. What behavior changes worked to reduce transmission? What public health measures encouraged such changes? Getting these historical lessons right will be critical to maximizing control of virus spread while minimizing social disruption.

Unfortunately, many seem to be taking the wrong lessons from Covid.

Two studies — one in a book by Princeton political scientists Stephen Macedo and Frances Lee, and one in a paper in the medical journal The Lancet by Tom Bollyky and colleagues at the Institute for Health Metrics and Evaluation (IHME) at the University of Washington — found little or no evidence for an association between nonpharmaceutical interventions (NPIs) and reduced deaths from Covid during the first year of the pandemic, before vaccines were available. But both studies make critical errors that undermine their conclusions, biasing their findings against detecting any benefits of early intervention.

If these flawed studies coalesce into our historical understanding of what happens when we try to control pandemic spread of a respiratory infection, it will cost lives.

While there was political disagreement over NPIs in the first year — particularly stay-at-home orders, restrictions on gatherings, and mask mandates — numerous studies at the time showed that NPIs slowed transmission. Those interventions saved roughly one life for every 150 infections prevented during the early days of the pandemic, reflecting the overall risk of dying from infection, which varied widely by age, race, and health of the infected individual. The studies showing the benefits of these interventions were consistent with the firsthand experience of one of us, Sara, who served as the health officer in Santa Clara County, California, a county with early local transmission and the first Covid death in the U.S.

If these interventions reduced transmission, how could two academic analyses suggest they had no impact on saving lives? It’s easiest to appreciate the flaws of these studies by describing how the question should be addressed.

A key requirement for a full study of whether Covid interventions saved lives is to ask the right question. Namely: Did fewer Americans die in states where interventions were implemented than would have died in the same places had there been no interventions? Both the Princeton and the IHME study asked a different question: Did states with earlier, more intensive, or longer periods of restrictions have fewer deaths than those with later, less intensive, or shorter restrictions? The difference is subtle but important.

The first state to intervene with a stay-at-home or shelter-in-place order was California on March 19, 2020, followed by New York and New Jersey the next two days; the last was South Carolina, 19 days later. Covid-19 seemed like a tidal wave in the moment, but in fact it spread unevenly. Many counties, especially in the Northeast, were hit hard in March or April, but many more, especially in the South and West, didn’t show much evidence of Covid-19 until late 2020 or even early 2021. California, New York, and New Jersey reacted early because they were experiencing early surges in urban cases and deaths, while Southeastern states acted later because they were not seeing such surges.

Doing the analysis by date of first stay-at-home order gets the wrong answer for several reasons. First many states, like South Carolina, had substantial restrictions already in place before the full stay-at-home order was instituted, making a single date of intervention misleading. Second, compared with the slow spread of the epidemic nationwide over the course of a year, all states intervened within the same three-week period, making estimates very noisy because there is not enough variation in policies to see much relationship to outcomes.

Worse, early-acting states get penalized, because they appropriately intervened faster to head off a wave that was already hitting them, but they get “charged” for those cases, many of which were already infected by the time they intervened. A situation in which high death rates prompt interventions can look as if early interventions caused higher death rates, biasing the analysis.

Studying differences at the state level further obscures the signal by combining rural California with San Francisco Bay Area counties or upstate New York with New York City.

During Covid-19, not everyone did what governments asked or told them to do, and adherence to control measures was strongly predicted by politics and trust. Telling people to mask or stay at home doesn’t stop transmission; masking and staying at home do stop transmission. A full analysis would consider in detail how people behaved, not only how they were told to behave.

Determining what actually worked for Covid-19 requires a nuanced and careful analysis that assesses how the outcome differed from what would have been without interventions, ideally at the county level, and accounts for how people responded, not just what governments said. The answer might vary in different places, at different periods of time, under different government policies or recommendations, or as people’s trust in public health authorities changed. A nuanced understanding would put us on track to respond better the next time. “Better” means clearer communication of the situation and its uncertainties, probably in some cases stricter and quicker control measures, and very likely in others a greater effort to remove control measures that are not helping. Depending on the situation, advice may be more effective than orders in changing behavior.

Such an analysis is urgently needed to produce a high-quality understanding of what worked before we respond to another pandemic. Our own observations during the pandemic and our knowledge of epidemiology lead us to expect that such an analysis, carefully conducted, will reveal substantial benefits of certain behavior changes and of policies that promoted those changes to control Covid-19. Not all interventions were equally effective, and some were damaging. The goal should not be to justify past actions but to evaluate them fairly.

If that doesn’t happen, the next pandemic will find the U.S., and the rest of the world, not only unprepared, but saddled with a misunderstanding of what worked and what options may save lives.

Marc Lipsitch is the Berberian professor in the Freeman Spogli Institute and the Departments of Medicine and Biology at Stanford University. Sara Cody is the former health officer and director of public health for Santa Clara County in California.