Breaking Down What COVID-19 Vaccine Effectiveness Means

The U.S. now has three safe and effective COVID-19 vaccines being shipped around the country and making it into people’s arms. All meet the U.S. Food and Drug Administration’s threshold for protecting people from COVID-19 disease. Yet two of them are about 95% efficacious, while another is 66% efficacious, which may make it tempting to rank them, and assume that people receiving the 66% efficacious shot are somehow less protected against COVID-19.

That’s not the case, however, since it’s not really possible to compare the vaccines to each other. In the table below, we’ve laid out what goes into that so-called vaccine efficacy number, and how you should evaluate the currently authorized vaccines:

To start, there’s a difference between efficacy and effectiveness. “Efficacy” refers to the results for how well a drug or vaccine works based on testing, while “effectiveness” refers to how well these products work in the real world, in a much larger group of people. Most people, however, use them interchangeably.

Next, it’s important to understand what these companies were actually measuring to come up with their efficacy numbers. In the case of the COVID-19 vaccines, the researchers were measuring how well their vaccines protected against symptoms of COVID-19. So their vaccine efficacy numbers refer to how well they lowered people’s chance of getting sick with COVID-19. Pfizer-BioNTech’s vaccine is 95% efficacious, meaning that for vaccinated people, it was 95% efficacious in protecting people from getting COVID-19 symptoms. It does not mean that 95% of people vaccinated won’t get COVID-19 and 5% will.

Similarly, Moderna’s vaccine is 94% efficacious, so it was 94% efficacious in protecting people from COVID-19, and Johnson and Johnson’s vaccine is 66% efficacious in doing the same.

But that doesn’t tell the whole story, for a few reasons.

First, the vaccines work in different ways. Pfizer-BioNTech and Moderna developed their vaccines using mRNA technology, which involves taking the genetic code for the spike protein of the SARS-CoV-2 virus and encasing it in a fat-based particle that is injected into the body. Once inside cells, those viral genes instruct immune cells to produce copies of the protein, which in turn switches the immune system into action, churning out cells like antibodies, among other activities. If the vaccinated person then gets infected with the actual virus, the body is ready to quickly produce those same antibodies that can stick to the virus and block it from infecting cells. Johnson & Johnson-Janssen’s vaccine uses a different strategy—a weakened cold virus that is reprogrammed to include the code for the spike protein. Once inside the body, the viral genes trigger a similar dedicated response against the virus. Because the vaccines use different ways to alert the immune system, the differing technologies could lead to varying degrees of efficacy.

Second, each vaccine has different dosing regimens. Pfizer-BioNTech’s and Moderna’s shots require two doses—in Pfizer-BioNTech’s case, the two shots are 21 days apart, and in Moderna’s case, they are 28 days apart. Janssen’s vaccine is a single shot. Pfizer-BioNTech and Moderna say their shots can stimulate the immune system after a single dose, but the second dose is needed to trigger the maximum response. Janssen’s scientists are also testing a booster dose of their vaccine to see if it too might amplify the immune response.

Third, the vaccine companies started recording COVID-19 symptoms at different times after vaccination to see how well their shots could prevent disease. In their respective late-stage studies, Pfizer-BioNTech started recording symptoms seven days after people received the second shot of its vaccine or placebo; Moderna started 14 days after the second dose; and Janssen recorded results from both 14 days and 28 days after its single dose.

Fourth, the vaccines were tested at different times. That’s important because of the genetic variants of the virus—some of which are more infectious than the original—that have emerged in recent months, after some companies had already completed initial testing of their vaccines. Pfizer-BioNTech and Moderna both tested their shots through most of 2020, when SARS-CoV-2 had not mutated as much, so most of the study participants who were infected, were infected by the same viral strain. Janssen, however, started its large phase 3 trial in September, and included people in the U.K., South Africa and Brazil, the three countries where new, mutated variants of the virus started to spread quickly. Indeed, many of the people in Janssen’s trial in those countries were infected with the new variants.

From a scientific point of view, that means Janssen’s study provided important clues about how well its vaccine could confront the growing threat of new variants of the virus. That could also explain why the Janssen vaccine’s efficacy is lower than those of the Pfizer-BioNTech and Moderna shots. In fact, studies have since shown that among people vaccinated with the Pfizer-BioNTech and Moderna vaccines, the level of antibodies against the South African variant in particular are up to six-fold lower compared to levels against the non mutated virus.

So for all of these reasons, comparing the three vaccines is a bit like comparing apples to oranges. What’s important is that all fight COVID-19 according to the FDA’s standard of efficacy, which is that vaccines should be at least 50% efficacious in protecting people from disease. And all are extremely effective in protecting people from getting severely ill with or dying from COVID-19, which, ultimately, is what we’d want any vaccine to do.