Topics Related to Prescription Drugs, Anxiety
and Depression
   Home      The Pharmaceutical Industry      Drug Study Problems

Drug Studies, or Why So Many Medical Studies Generate Wrong Answers

It's exciting to think that by taking a vitamin or herbal supplement that you can significantly reduce your risk of contracting  a disease or make yourself healthier. I'll make the connection to prescription drugs clearer later in this article.  If you've followed such studies in the last few years, you have reason to be cynical.  Vitamin E had its fifteen minutes of fame as a preventative for heart disease, an antioxidant that destroyed those bad ol' free radicals in your body and kept you healthy.  A study consisting of thousands of patients said so; the medical community said so.  They were wrong.[1]

The problem is that such studies appear on a regular basis and people believe them, act on them, and often, don't see the retraction.  But how can so many studies be so wrong?  The full answer would take a book-length Web site, but let me just mention some problems and perhaps make you a more critical reader of health claims.

The first thing to look for when you read, "Vitamin X Prevents Diabetes" is what animal the study is based on; that's right, what animal.  Often, usually buried in the text, the article will briefly describe the basis of the claim and it will be based on a "mouse model"-that is, vitamin X was given to a hundred mice specially bred to acquire diabetes and a hundred mice, also specially bred to acquire diabetes, were given placebo.  The result:  73%  of the mice receiving vitamin X acquired diabetes while 88% of the placebo-fed mice acquired diabetes.  So you have a 15% less chance of acquiring diabetes if you take vitamin X! 

Therefore  the proper title of the article might be, "If you are a specialty-bred mouse and if there is really a difference between 73% and 88% in two groups of a hundred mice each, you might have a 15% less chance of acquiring diabetes if you take vitamin X".  Not a very impressive title, you must admit-but there are two important points to be considered here.  The first is that the study was done on mice (The same sort of studies are done with drug candidates to see if a drug has therapeutic potential).  Now, mice are nice little mammals, like humans-much more interested in consuming cheese and peanut butter than the average human, but much like us, nonetheless. 

The problem is that lots of vitamins and drugs work really, really well on mice and have no effect, or a negative effect on humans.  I once asked a very  experienced scientist who did drug studies on mice why his mice studies so often did not translate into  having a positive effect on humans.  His answer was, "people aren't mice".  So if it's a study with just mice, don't automatically infer it has anything to do with the way humans will respond to a given chemical substance, be that substance a vitamin, herbal, or drug.  Think of the article as merely interesting.

The second objection to "Vitamin X Prevents Diabetes" is the meaning of the results.  Specialty-bred mice are really, really similar to each other-that is why mice studies are so popular; you can buy all sorts of specialty-bred mice-you name it, there's a mouse out there that has been bred for that trait.  So why weren't the results 100% of the mice that were given vitamin X diabetes-free and 100% of the placebo-fed mice diabetic?  Let's put this in a different, more homey, context.  Suppose you had two sons that went bowling and when they came home, they told you that one of them bowled a 100 and the other bowled an 120 game.  Would you conclude that the boy that bowled the 120 game was definitely a better bowler?-or would you be unsurprised if they went bowling again, that the "better bowler" might get a lower score this time out?  When given some thought, there are a lot of things that could explain those initial bowling scores, including randomness-so perhaps, a 20% difference just isn't impressive.

For a drug to be approved by the FDA for marketing, it must be more effective for the condition it attempts to treat than a placebo (usually a tablet that is made of exactly the same ingredients as the drug formulation, but not containing any drug).  "More effective" can mean just a few percent more effect-it just has to be "more effective".  There are a lot of drugs that are on the market that have had a very difficult time being at all more effective than placebo [2].  We'll return to this subject next time, but in summary of this article, remember when reading health claims to ask yourself the questions "what animal were the studies performed on" and "what was the percent difference between drug-dosed and placebo-dosed animals in terms of positive response-enough to convince me that the drug is likely to be helpful?"

[1] Can vitamin E protect your heart?

[2] Antidepressants no better than placebo?






 Content on this Web site  is for informational purposes only. We do not provide any medical advice, diagnosis or treatment