There was a recent article published in the British Medical Journal whose results suggested that calcium supplements increased heart attack risk. This was a meta-analysis combining 12 different studies with different groups of people, etc. The results of a meta-analysis are ALWAYS suspect, since they are a mathematical construct which may or may not represent the true state of affairs. The concept of combining several studies, none of which reached statistical significance, and re-analyzing them as a group, hoping the "law of large numbers", by combining them, would produce a scientific result was first advanced by Peto in several articles in the Lancet over 10 years ago, and I refer my more mathematically inclined readers to his papers. I was never convinced of the validity of his analysis and conclusions, and feel strongly that the results, if any, of a meta-analysis should be treated as a hypothesis to be tested rigorously by a proper, future medical study.
I discussed my mathematical objections at great length in this blog in my issue of May 30, 2009. The essential point is that the writer chooses which studies to include and chooses which of two methods he will use for analysis as to clinical significance: one depends on randomness (and hence the law of large numbers), and the other does not, and the two methods can easily give opposite results. I refer my interested readers back to the issue entitled"Analysis of Meta-Analysis, at www.ghthomas.blogspot.com.
I feel that it is unfortunate that authors of meta-analyses do not mention at the end of their articles the reasons that their particular analysis may be mathematically and statistically suspect.