February 2012: The Limits of VAERS

This month I want to discuss the limitations of VAERS data. Is it accurate? Is it useful?

Are the conclusions reached in these VAERS blog reports valid?

Let's start off by reviewing the two different ways that vaccine reactions can be monitored: active and passive.

An active monitoring system tracks a patient's entire history, starting before the vaccine is administered, and continuing even after a vaccine reaction. Follow-up interviews may be made after vaccination to increase thoroughness. Such systems are typically managed by a health-care provider and usually include the complete medical record of the patient.

A passive monitoring system (such as VAERS) is one in which reports of vaccine reactions are volunatarily submitted by patients, doctors, and pharmaceutical companies. No reports are filed for healthy people, very little past history is recorded, and few post-reaction follow-ups are available

Passive monitoring systems are less useful than active systems for a number of reasons:

• Underreporting. Because the reports are submitted voluntarily, many patients and doctors do not report vaccine reactions. Different estimates exist for the amount of underreporting and range from a factor of 10 to as much as a factor of 100 (meaning that the true number of vaccine reactions is between 10 and 100 times higher than what is reported to VAERS).
• Subject to bias. There are many issues that can bias a passive monitoring system. There might be more reports from new vaccines because people are worried about them. News articles might excite the public to submit more VAERS reports. It is possible that the kind of people who report reactions may be different than those who don't recognize a reaction or ignore it, causing more bias. And of course, serious reactions might be reported more frequently (even though they account for only 14% of VAERS data).
• Inaccuracy. Although most reports are submitted by medical professionals (83%, according to the government) there are still VAERS reports that are filed by people who are unfamiliar with vaccine reaction reporting, and therefore have incorrect or insufficient information. Some have suggested that the system is "gamed" by people intentionally filing false reports, but this is unlikely because the CDC and FDA staff screens the data and does follow-ups. For example, all VAERS reports are examined by trained medical coders in order to ensure correct symptom terminology. The May 2011 blog entry has more to say about errors in VAERS data.
• Missing context. Although the VAERS reporting form asks for previous medications and other history, it is not as complete as a full medical record. Also, because the report ends at the vaccine reaction, it misses any future medical issues that may arise. This is a problem because it is important to correlate "similar" patients when evaluating vaccine safety. For example, if one patient gets a vaccine and has a reaction, while another patient does not get a vaccine or reaction, you do not have enough information to say that the vaccine caused the reaction. What you need to do, in order to say anything with confidence, is to "stratify" the data, organizing patients into groups that have similar histories so that you can eliminate every difference except for the vaccination. Then, those who did not get the vaccine form a useful "control group" and can be compared with those who did get the vaccine. This is possible in an active monitoring system, but nearly impossible in a passive system.
• Missing rates. It is important to know the rate of vaccine reactions, so that you can say what percentage of vaccinated people react. Although the MedAlerts VAERS search engine does have biometric data showing the number of vaccine doses that were distributed, this is subject to many inaccuracies and is not as good as the data from an active monitoring system (see the July 2011 blog entry).

So what good is a passive monitoring system such as VAERS? Most people agree that VAERS is useful as an early-warning indicator of possible vaccine problems. Trends that show-up in VAERS may indicate a problem, which can be further investigated using other data.

Do these blog entries make appropriate use of the VAERS data? Some of the entries do exactly what VAERS is good at: they look through the data and identify possible trends. But other blog entries try to verify a trend by looking at VAERS data, and when this happens, the blog is careful in its wording, saying things like "VAERS data seems to confirm this trend" or "you might want to talk to your doctor about this." Some of the early blog entries (for example December 2009), do discuss issues that can affect the blog's conclusion.

The bottom line is this: We use VAERS because it's all we have. The only active vaccine monitoring system that exists in the U.S. is the Vaccine Safety Datalink, but it is government-private data that is available only to "qualified investigators". Even CDC medical researchers (who do have access to the Vaccine Safety Datalink) often use VAERS data instead. If these blog entries are faulty, then so are those government VAERS analyses that "prove" vaccine safety.

So I will continue to make VAERS blog entries each month and look at all aspects of the data. I hope that I do not mislead, but my goal is to raise awareness of potential vaccine safety issues. I am not a medical researcher (as I state quite clearly). In the end, some will disregard these blog entries because of the problems with VAERS, but hopefully others will find the issues raised here to be useful.

<< 1/2012: Seizures and 5-in-1 Vaccines 5/2012: The Anthrax Vaccine >>