I am sure that many if not all of you read or heard about the front page story in today's (Friday, May 27) NY Times that using Niacin to raise the level of HDL (the "good") cholesterol did NOT lower the incidence of heart disease. The false expectation of a benefit from raising HDL came from the common confusion (and wishful thinking, because doctors are always looking for ways to help their patients) between correlation and causation. Remember that just because taller people (on the average) weigh more than do short people, it does not therefore follow that the way to increase a person's weight is to increase his/her height.
We doctors have many suggestions to make to our patients about primary prevention of heart attacks . By primary, I mean prevention of the first heart attack. Since the biggest risk of having a heart attack resides in those who have already had one, we have been very aggressive and successful in finding ways to prevent a second heart attack (hence the phrase "secondary" prevention). This is why a patient who has been on no medicine at all and who enters the hospital with a heart attack usually goes home on five medicines, since each of them has been shown to greatly reduce the risk of having a second heart attack.
However, despite all our theories, and all our observations about commonalities in patients who have a first heart attack versus those "similar" patients who have not, very few interventions have been shown to reduce the risk of a first heart attack, and of these, stopping smoking leads the list. We also have over 30 years of strong evidence (but only in comparative studies) that mild intake of alcohol of any origin (beer, red or white wine, scotch, slivovits, etc.) reduces the relative and absolute risk of a first heart attack. Unfortunately, we have absolutely no idea how the C2H5OH molecule does this, and it certainly has not been linked to a reduction in cholesterol, but at autopsy mild alcohol ingesters have larger lumens (= less blockage) in their coronary arteries than do tee-totalers.We also have no strong evidence for the benefits of daily aspirin, and we certainly cannot explain the results of a 30 year epidemiological study published in Lancet as to why in the Far East patients with a higher BMI (i.e. heavier patients) had a reduced risk of heart attacks, nor can we explain the studies that demonstrate that patients who are somewhat overweight (BMI between 26 and 29) have the greatest survival rate if they have a heart attack. It is trivial to note that stopping smoking does not affect your total cholesterol, LDL cholesterol, or fasting glucose (except insofar as most ex-smokers gain 10 to 15 pounds the first year).
In approximate numbers, survivors of a first heart attack have at least a 10% chance of a repeat heart attack in the following two years, so the results of a pharmacological intervention are readily seen. But patients with a high cholesterol who have never had a heart attack have a comparative increased risk of a heart attack in the next TEN years of 10%, so the results of an intervention have to be stronger or longer to show a statistical benefit, which is why it is difficult to show the benefits of any primary prevention in any forward double-blind study. In fact, the first double blind study of cholesterol lowering, done on male smokers in Finland with gemfibrizol, showed no benefit of reducing heart attacks until the statistical method of analysis was altered. I do not have the space here that is necessary to demonstrate why it is statistically unsound to vary your method of statistical analysis if you don't like the result of the first analysis.
We are left with the result that elevated cholesterol, elevated LDL cholesterol and lowered HDL cholesterol are all statistically associated with an increased risk of a heart attack, but only lowering the LDL has been shown to lower the risk of heart attacks consistently. The Zetia study showed that Zetia lowered total cholesterol, but did not decrease the heart attack risk. (Which only means that this first Zetia study failed to show a benefit, and a second and possibly a third study would be needed to validate this result as a clinical conclusion.) It may well be that the real value in the use of statins to lower cholesterol lies in their anti-inflammatory properties. We now have a study that shows that raising HDL did not reduce the risk of heart attacks. We have also known for years that elevated triglycerides are a statistical risk factor for heart attacks, but no study as yet has shown that lowering the TGL level lowers the heart attack risk. (Some doctors empirically lower the TGL of all their diabetic patients because diabetics have an increased risk of heart attacks, and a relative insulin deficiency causes a rise in TGL, but there is no cause-and-effect proof yet.) We also definitely do not understand why many patients, especially females, can have total cholesterols of over 280 with no evidence of an increased heart attack risk in them or their mothers, and have very patent coronary arteries at autopsy, if one is done. We also don't know why (but we think we do) females have a lower risk for heart disease than do men, any more than we understand why lupus and anorexia are more common among women than men.
Again, we must be very careful of assuming that any and every statistical relationship is cause-and-effect, and that if elevated "A" is associated with disease "B" then pharmacologically lowering the value of "A" will lessen the incidence of disease "B". In other words, altering a statistical risk factor is not guaranteed to affect the incidence of the disease. Very few facts are "obvious" in medicine: just recall the universal advice to patients with calcium oxalate kidney stones to lower the amount of calcium in their diet to prevent a recurrence, until a physician at UTexas, Galveston showed conclusively that INcreasing the amount of calcium in the diet DEcreased the recurrence rate of calcium oxide kidney stones. It was also thought that anti-oxidants had a protective effect against cancer, but when 50% of a matched group of Finnish cigarette smokers were given daily doses of anti-oxidants, it was found that those who took daily anti-oxidants had an INcreased incidence of lung cancer, a result to which the American Cancer Society has not given proper importance.
Finally, we also have the Law of Unintended Consequences: When mothers throughout America gave their children pure bottled water to drink, there was an increase in the incidence of dental caries in children, until it was realized that bottled water contained no fluoride, so now they have returned to giving their children fluoride as prescription Poly-Vi-Flor fluoridated vitamins (or their equivalent), just as I did. And since they have lost the business of children with cavities, dentists are now pushing orthodontia and night guards in Medicare patients(!). In most states, when the mandatory drivers' seatbelt law was passed, the death rate for pedestrians INcreased in the first few years, and we still don't know why. And many, many doctors were guilty of prescribing combination estrogen-progesterone tablets for their post-menopausal female patients in an attempt to reduce their risk of a heart attack until the increased incidence of cancer was noted. When we advised patients to take daily Vitamin E (400 Units) on theoretical grounds to lower their heart attack risk, the first decent statistical analysis showed that daily pharmacological doses of Vitamin E INcreased the risk of having a heart attack. Similarly, when men with prostate cancer were given estrogen to decrease the effect of any androgen on the prostate, an increase in cerebral strokes was soon noted. And at the present, only God knows the effect of daily doses of CoEnzyme-Q, or the proper dosage if there is a benefit, or how many times a day it should be taken, if at all, and if so on an empty or a full stomach.
"Life is short, art long, opportunity fugitive, experimenting dangerous, reasoning difficult....."--- Hippocrates
Saturday, May 28, 2011
Saturday, May 14, 2011
The Problem With Medical Clinical "Studies"
I know that many readers are confused by or why one medical study refutes or contradicts a previous one,
or by how vehemently the purported results of a study are attacked by both physicians and patients who disagree with it. I will try to give some perspective and understanding about this, including why it happens. The practice of medicine is not a precise science, but rather a mixture of scientific principles, knowledge, art, intuition, luck,and a certain je ne sais quoi. In addition, you are dealing with people, all of whom have a tremendous number of "hidden variables", as Einstein would have put it.
In a well-designed scientific experiment, the scientist keeps all the (known) variables fixed except one, and then he/she systematically varies the one, and looks to see how the system changes. He is seeking data, or verification of a scientific law or principle, but he/she should also be aware of the possible existence of the unexpected, and be prepared to recognize it when it happens, as did Rutherford when some of the alpha particles he shot at a gold foil were deflected/reflected sharply, and he thereby inferred that the atom had a hard nucleus surrounded by a shell of atoms. When Galileo used the first telescope to examine the skies, he immediately detected the moons of Jupiter, whose existence was hitherto unexpected. When Jenner observed that farm girls who became infected with cowpox (which scarified but was not fatal) never developed smallpox, he hit upon the idea of vaccination, and showed that deliberate inoculation with the fluid from the pustules of patients with cowpox prevented the recipients from infection with smallpox. (Of course, this was done years before Institutional Review Boards and the doctrine of Informed Consent had taken hold.)
No one argues about the correct value for the speed of light, or Planck's Constant, or the diameter of the Earth, or the distance from the earth to the sun or that the core of the earth is liquid, or that the earth is round. (Actually there is a Flat Earth Society headquartered in London, but they have yet to construct a viable model of a flat earth than explains all modern observations.) What I am saying is that if there is vigorous, heated argument about a "fact", then either the data supporting the "fact" is not conclusive, or that some people have a strong vested interest or personal or political belief that makes the existence of the "fact" inconvenient for their belief system or is psychologically stressful. The concept that "ye shall know the truth and the truth shall make ye free" has been replaced by the concept that either the truth is not for all people (hence censorship), or that the truth might lead your thought patterns into dangerous realms (hence the insistence in the Middle Ages that the Earth was at the center of the universe, or the execution of Socrates by the democratic city of Athens for "leading the youth away from the true gods", or the removal of all mention of Trotsky from the Soviet Encyclopedia by Stalin). I would briefly note the vociferous reaction to the announcement by one research group that according to their studies, annual mammograms of women between the ages of 40 to 49 did not seem to save lives from breast cancer as well as the present arguments for and against lowering the amount of salt in the diet. In medicine we still have the ongoing argument between Alcoholics Anonymous who insist that the only treatment for alcoholism is total abstinence, and other drug counselors who say that it is safe to reduce alcoholics to one or two drinks a day. Committees keep on lowering the "normal"value of fasting glucose, although no one has demonstrated any benefit in either reduced morbidity or reduced mortality by so doing.
Now in physics we worked with inanimate objects, which we could construct as being identical and had no free will or volition. In medicine, we work with humans of all ages, sexes, color, heights and weights, with free will and various cultural and religious beliefs as well as various degrees of compliance with advice and/or medications, who may be taking some unknown herbal medicine or vitamin supplement, so the definition of "identical" is quite different. (And remember that vitamins are a huge variable: they are not under FDA jurisdiction, and the labeled dose may well be incorrect. Several years ago The Medical Letter chemically measured the amount of Vitamin E in gelcaps labeled "Vitamin E 400 Units". They found the amount of vitamin E in caps from different manufacturers varied from zero to 1200 Units.) I will not discuss Hume's famous analysis of the fallacies inherent in the inductive method of reasoning, but I will note that retrospective studies generate inductive "laws" or suggestions, while double-blind studies, assuming the model is correct, are essentially generating deductive solutions. And overall, both types of studies must deal with the question of their statistical accuracy, so I always tell my patients to wait for the second clinical study, which very often contradicts the first. I have discussed the problems with and limitations of meta-analyses in two previous blogs, so I will not discuss that here, except to reiterate that IMHO the results of any meta-analysis should be used to suggest a future double-blind experiment, and not to suggest a medical treatment.
I will not discuss the proper application of statistics to clinical studies, since that would make this blog much too long. The most common error that retrospective and case control studies make is to mistake correlation for causation. The studies may show that patients with disease X have diet Y or do activity Z, but very often when we then do a controlled study and take a group of (hopefully) identical patients, and have half of them adopt diet Y or activity Z we find that this intervention does not affect the incidence of disease X. For instance, no forward study has ever demonstrated that a low fat diet reduces the incidence of cancer (which of course has not prevented the American Cancer Society from advocating a low fat diet as a cancer preventative). Similarly, no one has demonstrated that a low salt diet prevents high blood pressure or heart failure or heart attacks in healthy individuals. And we still don't know if the optimum diet is one, two, three, four or more meals a day, and whether or not the largest meal should be eaten in the AM, at noon, or in the PM. And once a recommendation attains the force of law, it is rarely rescinded. For instance many studies have shown that a pregnant woman can safely have one drink a day without adverse effects on the fetus, but the Surgeon General's warning remains on every bottle of alcohol sold in the U.S. We also have been told that reducing cholesterol reduces the rate of heart attacks, but cannot explain why Zetia, which reduced cholesterol, did not appear to provide this benefit.
One example of the interpretation of treatment or intervention is the history of tuberculosis in the 20th century. The incidence and death rate from TB started to decrease in the 1920's in the USA, even though we had no antibiotics that worked at that time. Some social reformers claimed that it was due to the reduction in crowding and filth secondary to the elimination of many tenements, but of course this could neither be proved nor disproved with the available data. Dr. Waksman discovered injectable streptomycin, which was the first anti-TB drug that worked. It had the unfortunate side effect of causing irreversable damage to the 8th cranial nerve, leading to severe deafness in the patients so treated. Then INH, or isoniazid was developed in the laboratory. One of the largest groups of patients treated were veterans in the Manhattan VA hospital with TB: 50% received streptomycin and 50% received INH. INH was shown to be superior to streptomycin with the added advantage of not causing deafness. The old-time medicine cabinets had a slot in the rear for the disposal of disposable razor blades. When they tore down the old VA hospital, thousands of INH tablets were found behind the medicine cabinets, so the INH was probably more effective than anyone dreamed.
The problem of course with all medical studies is that in successive studies the groups are never identical, so that for instance in a double blind study of any treatment of children in the 50's we had one group of children, while the identical study done in the 70's would be done on children who had received fluoride to prevent tooth decay. Therefore there is no reason to expect a priori that successive medical intervention studies should show the same result, or that a study in London should give the same result as a study in San Francisco. The laws of physics are invariant under a Galilean transformation, but people are not. The same study done in an equatorial country and in a northern European country may disagree for the same reason that children who grow up in the equatorial zones have a significantly different incidence of multiple sclerosis than do children who grow up in temperate zones (as do females who did and did not have a cat as a pet before the age of 16). For all we know, we should do historical genetic testing on DNA from cheek swabs because patients whose distant ancestors came from Asia Minor may have a different response to disease and treatment than those whose ancestors came from the steppes of Russia. Or perhaps nearsighted people and farsighted people also have different disease coping mechanisms or immune responses because of linkage on the same chromosome, and no one allows for that.No two people are identical biologically, genetically, or in their immune defenses or in their propensity to develop diabetes, or heart disease or cancer. We even have the false belief that a teacher can teach a group of 35 first graders as well as she can teach a class of 15.
Finally the placebo effect which has been shown to be as high as 30% is nowhere accounted for in either the control group or the treated group, and the kappa statistic is an unreliable method for detecting the effect of random coincidence, since the effect can sometimes calculate out as being greater than one, which violates its basic statistical assumption.
In the words of Ralph Waldo Emerson: "No facts to me are sacred, none are profane. I simply experiment and observe......No truth is so sublime that it may not become trivial tomorrow in the light of new thoughts and facts".
or by how vehemently the purported results of a study are attacked by both physicians and patients who disagree with it. I will try to give some perspective and understanding about this, including why it happens. The practice of medicine is not a precise science, but rather a mixture of scientific principles, knowledge, art, intuition, luck,and a certain je ne sais quoi. In addition, you are dealing with people, all of whom have a tremendous number of "hidden variables", as Einstein would have put it.
In a well-designed scientific experiment, the scientist keeps all the (known) variables fixed except one, and then he/she systematically varies the one, and looks to see how the system changes. He is seeking data, or verification of a scientific law or principle, but he/she should also be aware of the possible existence of the unexpected, and be prepared to recognize it when it happens, as did Rutherford when some of the alpha particles he shot at a gold foil were deflected/reflected sharply, and he thereby inferred that the atom had a hard nucleus surrounded by a shell of atoms. When Galileo used the first telescope to examine the skies, he immediately detected the moons of Jupiter, whose existence was hitherto unexpected. When Jenner observed that farm girls who became infected with cowpox (which scarified but was not fatal) never developed smallpox, he hit upon the idea of vaccination, and showed that deliberate inoculation with the fluid from the pustules of patients with cowpox prevented the recipients from infection with smallpox. (Of course, this was done years before Institutional Review Boards and the doctrine of Informed Consent had taken hold.)
No one argues about the correct value for the speed of light, or Planck's Constant, or the diameter of the Earth, or the distance from the earth to the sun or that the core of the earth is liquid, or that the earth is round. (Actually there is a Flat Earth Society headquartered in London, but they have yet to construct a viable model of a flat earth than explains all modern observations.) What I am saying is that if there is vigorous, heated argument about a "fact", then either the data supporting the "fact" is not conclusive, or that some people have a strong vested interest or personal or political belief that makes the existence of the "fact" inconvenient for their belief system or is psychologically stressful. The concept that "ye shall know the truth and the truth shall make ye free" has been replaced by the concept that either the truth is not for all people (hence censorship), or that the truth might lead your thought patterns into dangerous realms (hence the insistence in the Middle Ages that the Earth was at the center of the universe, or the execution of Socrates by the democratic city of Athens for "leading the youth away from the true gods", or the removal of all mention of Trotsky from the Soviet Encyclopedia by Stalin). I would briefly note the vociferous reaction to the announcement by one research group that according to their studies, annual mammograms of women between the ages of 40 to 49 did not seem to save lives from breast cancer as well as the present arguments for and against lowering the amount of salt in the diet. In medicine we still have the ongoing argument between Alcoholics Anonymous who insist that the only treatment for alcoholism is total abstinence, and other drug counselors who say that it is safe to reduce alcoholics to one or two drinks a day. Committees keep on lowering the "normal"value of fasting glucose, although no one has demonstrated any benefit in either reduced morbidity or reduced mortality by so doing.
Now in physics we worked with inanimate objects, which we could construct as being identical and had no free will or volition. In medicine, we work with humans of all ages, sexes, color, heights and weights, with free will and various cultural and religious beliefs as well as various degrees of compliance with advice and/or medications, who may be taking some unknown herbal medicine or vitamin supplement, so the definition of "identical" is quite different. (And remember that vitamins are a huge variable: they are not under FDA jurisdiction, and the labeled dose may well be incorrect. Several years ago The Medical Letter chemically measured the amount of Vitamin E in gelcaps labeled "Vitamin E 400 Units". They found the amount of vitamin E in caps from different manufacturers varied from zero to 1200 Units.) I will not discuss Hume's famous analysis of the fallacies inherent in the inductive method of reasoning, but I will note that retrospective studies generate inductive "laws" or suggestions, while double-blind studies, assuming the model is correct, are essentially generating deductive solutions. And overall, both types of studies must deal with the question of their statistical accuracy, so I always tell my patients to wait for the second clinical study, which very often contradicts the first. I have discussed the problems with and limitations of meta-analyses in two previous blogs, so I will not discuss that here, except to reiterate that IMHO the results of any meta-analysis should be used to suggest a future double-blind experiment, and not to suggest a medical treatment.
I will not discuss the proper application of statistics to clinical studies, since that would make this blog much too long. The most common error that retrospective and case control studies make is to mistake correlation for causation. The studies may show that patients with disease X have diet Y or do activity Z, but very often when we then do a controlled study and take a group of (hopefully) identical patients, and have half of them adopt diet Y or activity Z we find that this intervention does not affect the incidence of disease X. For instance, no forward study has ever demonstrated that a low fat diet reduces the incidence of cancer (which of course has not prevented the American Cancer Society from advocating a low fat diet as a cancer preventative). Similarly, no one has demonstrated that a low salt diet prevents high blood pressure or heart failure or heart attacks in healthy individuals. And we still don't know if the optimum diet is one, two, three, four or more meals a day, and whether or not the largest meal should be eaten in the AM, at noon, or in the PM. And once a recommendation attains the force of law, it is rarely rescinded. For instance many studies have shown that a pregnant woman can safely have one drink a day without adverse effects on the fetus, but the Surgeon General's warning remains on every bottle of alcohol sold in the U.S. We also have been told that reducing cholesterol reduces the rate of heart attacks, but cannot explain why Zetia, which reduced cholesterol, did not appear to provide this benefit.
One example of the interpretation of treatment or intervention is the history of tuberculosis in the 20th century. The incidence and death rate from TB started to decrease in the 1920's in the USA, even though we had no antibiotics that worked at that time. Some social reformers claimed that it was due to the reduction in crowding and filth secondary to the elimination of many tenements, but of course this could neither be proved nor disproved with the available data. Dr. Waksman discovered injectable streptomycin, which was the first anti-TB drug that worked. It had the unfortunate side effect of causing irreversable damage to the 8th cranial nerve, leading to severe deafness in the patients so treated. Then INH, or isoniazid was developed in the laboratory. One of the largest groups of patients treated were veterans in the Manhattan VA hospital with TB: 50% received streptomycin and 50% received INH. INH was shown to be superior to streptomycin with the added advantage of not causing deafness. The old-time medicine cabinets had a slot in the rear for the disposal of disposable razor blades. When they tore down the old VA hospital, thousands of INH tablets were found behind the medicine cabinets, so the INH was probably more effective than anyone dreamed.
The problem of course with all medical studies is that in successive studies the groups are never identical, so that for instance in a double blind study of any treatment of children in the 50's we had one group of children, while the identical study done in the 70's would be done on children who had received fluoride to prevent tooth decay. Therefore there is no reason to expect a priori that successive medical intervention studies should show the same result, or that a study in London should give the same result as a study in San Francisco. The laws of physics are invariant under a Galilean transformation, but people are not. The same study done in an equatorial country and in a northern European country may disagree for the same reason that children who grow up in the equatorial zones have a significantly different incidence of multiple sclerosis than do children who grow up in temperate zones (as do females who did and did not have a cat as a pet before the age of 16). For all we know, we should do historical genetic testing on DNA from cheek swabs because patients whose distant ancestors came from Asia Minor may have a different response to disease and treatment than those whose ancestors came from the steppes of Russia. Or perhaps nearsighted people and farsighted people also have different disease coping mechanisms or immune responses because of linkage on the same chromosome, and no one allows for that.No two people are identical biologically, genetically, or in their immune defenses or in their propensity to develop diabetes, or heart disease or cancer. We even have the false belief that a teacher can teach a group of 35 first graders as well as she can teach a class of 15.
Finally the placebo effect which has been shown to be as high as 30% is nowhere accounted for in either the control group or the treated group, and the kappa statistic is an unreliable method for detecting the effect of random coincidence, since the effect can sometimes calculate out as being greater than one, which violates its basic statistical assumption.
In the words of Ralph Waldo Emerson: "No facts to me are sacred, none are profane. I simply experiment and observe......No truth is so sublime that it may not become trivial tomorrow in the light of new thoughts and facts".
Subscribe to:
Posts (Atom)