I enjoyed the 'Never vaccinated' vs 'Ever vaccinated' mortality rate illusion substack by Norman Fenton. It includes a short video explaining how survivor or survivorship bias can lead to massively exaggerated claims of vaccine efficacy and safety. As it happens, I am currently experimenting with explainer video production, so I thought I could create a similar video, using a different example. Here is the result:
Of course, the example provided is extremely simple to simplify the calculations. But the effect of these simplifications does not invalidate the conclusions:
That the vaccination progresses linearly or not is irrelevant. As shown in the video, the important issue is that we should analyse infections considering the time spent by the individuals taking part in the study in each estate: vaccinated and unvaccinated. For that reason, observational studies need to use the concept of person years to estimate the effect of interventions.
A constant rate of infection is unrealistic, but the reality is even worse. Vaccination took place in many countries during periods in which the rate of infection was maximum. As a result, the overestimation of the vaccine effectivity would be even higher using real data than what I show on the video.
I have considered no age variation in the vaccination and infection rates. This is a serious limitation of the exercise, but many studies I have seen share this limitation.
Finally, I worked with the concept of infections rather than deaths. This simplifies enormously the explanation. The bias introduced by not considering deaths is small if the mortality rate is small (as it is the case for covid). This is my first attempt at using the method of explainer videos to support the messages in this blog. For that reason, I really appreciate any feedback regarding whether you find the video instructive and understandable.
Note: The video uses an artificial voice created with 11ElevenLabs using the free tier.