As a former research physicist and editor and reviewer for physics journals, as well as a reviewer for 10 years for the Annals of Internal Medicine, I feel compelled to comment on the plethora of clinical medical studies that are being released faster and more furiously. Research scientists understand the limitations inherent in clinical reports, whether published in a journal or presented verbally at a conference, but people who get their information by reading internet or newspaper summaries of the results do not. I will try to outline what to look for in these reports, and what level of trust should be placed in the reported results, without resorting to scientific and statistical jargon such as regression to the mean, Simpson's Paradox, and the Law of Large Numbers. Interested readers might want to read my two earlier blogs on the fallacies inherent in using meta-analyses to suggest valid clinical treatments.
One problem in reading newspaper summaries of published articles is that in their efforts to simplify the results and conclusions for their readers, columnists have made interpretive errors. At the very least, the columnist should refer you to the National Library of Medicine's web page, www.PubMed.gov, where the abstract of the discussed article is contained. As a general rule, the abstract will contain the essential conclusions of the published journal article. There are a number of reputable newspaper columnists who are themselves practicing doctors, such as Danielle Ofri, Perri Klass, and Lisa Goldman, and I have always found their columns to be accurate. Without mentioning names, I have found errors in most articles written by science writers, especially when the interpretation of the published results is open to question, or ambiguous.
You may take it as a general rule that if there is heated argument about the "correct" interpretation of a scientific result, then the clinical interpretation is not yet known with certainty. In physics we would say that if you have to argue about the interpretation of data points, then more data is needed. And let us not forget the government's insidious modification of established facts, much as Stalin removed the faces of Politburo members from the Soviet Encyclopedia if they fell out of favor. The best example here in the U.S. deals with the iconic picture of the abstract painter Jackson Pollock. This well-known photograph showed him bent over his work with his ever-present cigarette dangling from his mouth. When the U.S. Postal Service reproduced this picture to use on a commemorative stamp, his cigarette was airbrushed out of the picture.
For starters, please remember that the gold standard for the imprimatur of the probable accuracy of a scientific or clinical report is its publication in a journal article that has been reviewed by a scientist in the field. This has been shown to be the best way to make sure that there are no biases in the article, that the data is presented clearly and fairly, and that the conclusions follow from the data and are clinically significant.
Ideally, if any graphs are presented with data points, the data points should have error bars attached to them, showing the variation that could have been introduced by the imprecision of their measurement. Also, the authors of the article should report who paid for the research and if they have received speaking fees or consultation fees from the government or drug companies, or any other sources (which of course newspaper reporters and TV news commentators almost never report about themselves).In addition, if the authors speculate about theoretical and possible future applications of their clinical results, the ideas should clearly be labeled as speculations and suggestions.
From what I have said, it should be clear that a verbal report given at a medical conference should only be treated as an interesting idea. All too often, initial verbal reports or announcements at news conference do not withstand the rigor of being reviewed for journal publication, let alone the test of time. I'm sure we all remember the foofraw connected with the report of table-top cold fusion. In fact, in this day and age of instant celebrity, I would expect any discoverer of an important clinical event to appear on the Oprah show (and this is meant as a compliment to Oprah, and not a denigration of her show). For this reason I take all medical news reports with a grain of salt until I can read about it in a refereed journal. In fact, The New England Journal of Medicine will often publish two similar studies in the same issue, and then have an editorial comment on why the two agree or disagree.
And even reputable journal articles may not be correct. All too often one article is followed shortly (and sometimes in the same journal!) with a clinical research article coming to the opposite conclusion. To mention just a few: is lowering salt in the diet of an average person beneficial or harmful? do mammograms of women in their 40's save lives from breast cancer? does treating prostate cancer save lives? And then we come to the famous conflict in presenting data between relative risk and absolute risk. If the death rate from lightning strikes is 1/2,000,000 and I can reduce it to 1/1,000,000, I can present this result either by saying that I reduced your (relative) risk of lightning-stroke death by 50%, or I reduced your (absolute risk) by 1:1,000,000. Most journal articles and reports speak of the benefits of a new medical treatment in relative risk, because that number is always larger, and sounds much more impressive. A 35% reduction in your change of getting leprosy would probably not be of much interest or medical importance to you because your absolute risk is so low. OTOH you might leap to take a new treatment that reduced your chance of getting a heart attack by 35%, but is it a 35 % reduction over the next year, or is it a 35% reduction over the next 10 years, i.e. only a 3.5% reduction per year. In other words, how long do you have to live to reap the quoted benefit of the proposed treatment?
Let me close with a statement attributed to Voltaire: No one argues over whether or not 7 x 8 = 56, but Giordano Bruno was burned at the stake for maintaining that the earth moved around the sun.
Showing posts with label medical research. Show all posts
Showing posts with label medical research. Show all posts
Thursday, November 29, 2012
Friday, February 4, 2011
Medical Research Errors, or Don't Believe all that you Read
I did physics research for over 10 years, and clinical medical research for over 25 years. I have reviewed physics papers (as assistant editor of a physics journal) and medical papers, especially for the Annals of Internal Medicine. As a physics reviewer, I could often suggest revisions that would make the paper publishable, but as a medical reviewer I found this to be impossible. During my 10 year tenure as a reviewer in my medical field, I would estimate that I rejected 90% of the papers I reviewed, usually for lack of scientific rigor, and my negative review was never overridden by the article being published in a medical journal of similar significance.
There are various problems with scientific rigor in medical journals, but the most serious is their improper use of statistics, followed by lack of reproducibility, i.e. other researchers cannot replicate the results. In science, this is usually due to fraud (black ink spots placed on mice) or self-delusion (N-rays in France), but in medicine it is usually a sign that the first result was not entirely correct. I always tell my patients who rush in breathlessly with the latest report of medical research to wait for the second and confirming report. The second report often does not confirm the first. I also tell them to beware of medical studies that look for surrogate endpoints (lowering of blood pressure or cholesterol) rather than significant clinical endpoints(stroke, heart attack).
There are too many books, articles, and TV shows including some authored by doctors, that are studded with phrases such as "the evidence suggests", "it seems reasonable that", "past history shows" or all patients with or without condition X did or did not have disease Y. None of these statements contains a scientific truth, and such phrases would never appear in a physics or chemistry journal. A hard fact is rarely stated, such as that 500mg 2 x day of Vitamin C will lower your uric acid level by approximately 1.0 and thereby reduce your risk of a gout attack (this latter statement has been substantiated by research). We have very little good clinical evidence about many medical conditions, although it is almost universally agreed that antibiotics can kill bacteria and save lives, and that vaccinations are beneficial.
We still don't understand why only 10% of smokers get lung cancer (although we know with 100% certainty that all smokers will develop emphysema if they smoke long enough---40 pack years is usually sufficient). We don't understand why different patients with high blood pressure develop problems in different organs: stroke, kidney failure, heart attack. We don't understand why one alcoholic gets cirrhosis of the liver, one dies of hemorrhagic pancreatitis, and one gets the Korsakoff Syndrome. Similarly, we cannot predict which organ will fail in a patient with insulin-dependent diabetes, or if in fact any of them will.
The main problem, of course, is that each patient is an individual biochemical laboratory, and no two patients react the same way to any medical intervention. This is always true if the patient is on three or more drugs, since almost all pharmacological studies are done on patients on no drugs. (And no one knows how all the over-the-counter health foods interact with anything.) Many medical recommendations are based on three or fewer studies, while there are over 50 verifications of Bell's Inequality in quantum physics, and hundreds of measurements of the speed of light. Furthermore the results of a medical study apply only statistically, in that not everyone is cured of strep throat by penicillin, and some people with strep throat recover without any antibiotic at all. If I were responsible for the treatment of 1,000,000 patients, I would have no hesitation in making certain recommendations, but if I am recommending treatment for just one person, I am never 100% certain if the treatment (a) will work, and (b) is necessary.
In a previous blog I carefully reviewed the mathematical and statistical flaws inherent in all meta-analyses, so let me just say here that every result suggested by any meta-analysis should be verified by double-blinded random testing, despite the rapidity with which the media trumpet the results of the study.
I should also mention the problem of confounding conditions. When we do a clinical intervention study, we try to make the treated and control patients physiologically as similar as possible. When we do not, we come to the false conclusion that coffee drinkers suffer more heart attacks than do non-drinkers, until it was realized that coffee drinkers also smoke more cigarettes than the non-drinkers. We still have conflicting studies on whether tall people live longer or shorter lives than short people, and are similarly lacking in knowledge about righties versus lefties. (But I do wonder why all left-handed tennis players seem to have a natural topspin on their forehand on television.) Many men gleefully reported to their wives and significant others the one study that seemed to indicate that frequent ejaculations had a protective effect against prostate cancer, but many fewer people and organizations trumpet the many studies that show that mild alcohol ingestion is protective against heart attacks, and the fact that NO study has shown that mild alcohol ingestion contributes to heart attacks.
And please ignore all studies reported at all medical conferences. Until these studies have been written up, reviewed and then published, there is no way for anyone to verify their accuracy. And beware doubly of the placebo effect: In the 1950's, before we had ethical treatment review boards, 12 patients with angina had open heart surgery (I will not name the surgeon or the institution). 6 patients had the "real" surgery, and the other 6 had "sham" surgery, in that they only had their sternum (chest) split open, and then were wired shut again without anything being done to their hearts. 4 out of 6 in each group had their anginal pain and symptoms relieved.
What I am trying to say is that doctors overstate the degree of medical certainty behind many of their medical prescriptions and suggestions. Remember that before 1973 homosexuality was a psychiatric disorder, and in the new DSM-V manual you will find that narcissism will no longer be one. (Although I expect that if a drug company develops a medicine to treat narcissism it will quickly become a disease again.)
Finally, if your internet searches seem to indicate that some doctor or institution has a "special" treatment for a disease, please remember that all doctors want to cure their patients, and many would not mind becoming famous. If someone had a new cure for lung cancer, or rheumatoid arthritis, it would not be kept secret, and doctors throughout the world would know about it. There is one case of medical secrecy in history: A family of French Huguenot doctors, the Chamberlens fled to England in the 16th century. One of them invented obstetrical forceps to aid with difficult births. (Remember that C-sections were almost uniformly fatal in those days.) They became famous for their work with difficult deliveries, and they kept the existence of obstetrical forceps secret for over 150 years. I do not think that this would be possible now.
There are various problems with scientific rigor in medical journals, but the most serious is their improper use of statistics, followed by lack of reproducibility, i.e. other researchers cannot replicate the results. In science, this is usually due to fraud (black ink spots placed on mice) or self-delusion (N-rays in France), but in medicine it is usually a sign that the first result was not entirely correct. I always tell my patients who rush in breathlessly with the latest report of medical research to wait for the second and confirming report. The second report often does not confirm the first. I also tell them to beware of medical studies that look for surrogate endpoints (lowering of blood pressure or cholesterol) rather than significant clinical endpoints(stroke, heart attack).
There are too many books, articles, and TV shows including some authored by doctors, that are studded with phrases such as "the evidence suggests", "it seems reasonable that", "past history shows" or all patients with or without condition X did or did not have disease Y. None of these statements contains a scientific truth, and such phrases would never appear in a physics or chemistry journal. A hard fact is rarely stated, such as that 500mg 2 x day of Vitamin C will lower your uric acid level by approximately 1.0 and thereby reduce your risk of a gout attack (this latter statement has been substantiated by research). We have very little good clinical evidence about many medical conditions, although it is almost universally agreed that antibiotics can kill bacteria and save lives, and that vaccinations are beneficial.
We still don't understand why only 10% of smokers get lung cancer (although we know with 100% certainty that all smokers will develop emphysema if they smoke long enough---40 pack years is usually sufficient). We don't understand why different patients with high blood pressure develop problems in different organs: stroke, kidney failure, heart attack. We don't understand why one alcoholic gets cirrhosis of the liver, one dies of hemorrhagic pancreatitis, and one gets the Korsakoff Syndrome. Similarly, we cannot predict which organ will fail in a patient with insulin-dependent diabetes, or if in fact any of them will.
The main problem, of course, is that each patient is an individual biochemical laboratory, and no two patients react the same way to any medical intervention. This is always true if the patient is on three or more drugs, since almost all pharmacological studies are done on patients on no drugs. (And no one knows how all the over-the-counter health foods interact with anything.) Many medical recommendations are based on three or fewer studies, while there are over 50 verifications of Bell's Inequality in quantum physics, and hundreds of measurements of the speed of light. Furthermore the results of a medical study apply only statistically, in that not everyone is cured of strep throat by penicillin, and some people with strep throat recover without any antibiotic at all. If I were responsible for the treatment of 1,000,000 patients, I would have no hesitation in making certain recommendations, but if I am recommending treatment for just one person, I am never 100% certain if the treatment (a) will work, and (b) is necessary.
In a previous blog I carefully reviewed the mathematical and statistical flaws inherent in all meta-analyses, so let me just say here that every result suggested by any meta-analysis should be verified by double-blinded random testing, despite the rapidity with which the media trumpet the results of the study.
I should also mention the problem of confounding conditions. When we do a clinical intervention study, we try to make the treated and control patients physiologically as similar as possible. When we do not, we come to the false conclusion that coffee drinkers suffer more heart attacks than do non-drinkers, until it was realized that coffee drinkers also smoke more cigarettes than the non-drinkers. We still have conflicting studies on whether tall people live longer or shorter lives than short people, and are similarly lacking in knowledge about righties versus lefties. (But I do wonder why all left-handed tennis players seem to have a natural topspin on their forehand on television.) Many men gleefully reported to their wives and significant others the one study that seemed to indicate that frequent ejaculations had a protective effect against prostate cancer, but many fewer people and organizations trumpet the many studies that show that mild alcohol ingestion is protective against heart attacks, and the fact that NO study has shown that mild alcohol ingestion contributes to heart attacks.
And please ignore all studies reported at all medical conferences. Until these studies have been written up, reviewed and then published, there is no way for anyone to verify their accuracy. And beware doubly of the placebo effect: In the 1950's, before we had ethical treatment review boards, 12 patients with angina had open heart surgery (I will not name the surgeon or the institution). 6 patients had the "real" surgery, and the other 6 had "sham" surgery, in that they only had their sternum (chest) split open, and then were wired shut again without anything being done to their hearts. 4 out of 6 in each group had their anginal pain and symptoms relieved.
What I am trying to say is that doctors overstate the degree of medical certainty behind many of their medical prescriptions and suggestions. Remember that before 1973 homosexuality was a psychiatric disorder, and in the new DSM-V manual you will find that narcissism will no longer be one. (Although I expect that if a drug company develops a medicine to treat narcissism it will quickly become a disease again.)
Finally, if your internet searches seem to indicate that some doctor or institution has a "special" treatment for a disease, please remember that all doctors want to cure their patients, and many would not mind becoming famous. If someone had a new cure for lung cancer, or rheumatoid arthritis, it would not be kept secret, and doctors throughout the world would know about it. There is one case of medical secrecy in history: A family of French Huguenot doctors, the Chamberlens fled to England in the 16th century. One of them invented obstetrical forceps to aid with difficult births. (Remember that C-sections were almost uniformly fatal in those days.) They became famous for their work with difficult deliveries, and they kept the existence of obstetrical forceps secret for over 150 years. I do not think that this would be possible now.
Friday, August 7, 2009
Medical Statistics, Research and News
As a former experimental physicist who did funded research for several years, published many papers in physics journals, and was on the editorial board of a physics journal for three years, I have several comments about the quality of medical publications and the interpretation of their results by the lay press.
First, it seems to be a common habit to publish data points in a graph without error bars. This makes it impossible to interpret the results properly. (For lay persons, the error bars show the range of +/- 2 standard deviations, which means that IF the data and error distribution is Gaussian, then there is a 2.5% probability that a repeat measurement would be above and a 2.5% probability of being below the range of the error bars.) Then this mistake is often compounded by connecting the data points by a sequential series of straight lines, rather than a French curve or a least squares fit.
When a result is presented as being "statistically significant", what is meant is that there is only a 5% probability of the result being incorrect (yes, I know I am simplifying here). However, a statistically significant result may not be clinically significant. It is easy to demonstrate that if you have enough subjects, something will be statistically significant. But is it really medically useful to know how to decrease your risk of being killed by a falling meteorite by 50%?
The lay press is also woefully ignorant of the concept of statistical variation. If you tabulate, for instance, cases of breast cancer in every county in a state, one county has to be the highest, and one has to be the lowest, without any "cause". Every time there is a clustering of cases (as in lymphoma in Passaic, N.J. about 15 years ago), there is a rush to find the cause.
Then the relative risk rather than absolute risk is emphasized. Again, if your chance by being killed by a falling meteorite is one in a million, then if I decrease your relative risk by 50%, I have only lowered your absolute risk by 0.0001%.
Finally, and this is the most egregious mistake of all, is the use of surrogate endpoints. For instance, in the study of the effect of lowering cholesterol by the use of Zetia, instead of looking at heart attacks or stroke as a primary endpoint, the thickness of the intima of the carotid artery was used as a surrogate endpoint. If the most common cause of arterial blockage is rupture of a plaque rather than embolic, then this surrogate endpoint is not medically useful. (Personally, I think the evidence points to the anti-inflammatory effects of ASA and statins as reducing the risk of plaque rupture and acute blockage, but no one has yet been able to detect such an acute event when it happens in humans.)
Actually, in fairness to medical researchers, I should mention the main limitation that they face. In physics, if an experimental result is published, other researchers rush to try to repeat the result by a different experimental technique, using different apparatus, to help establish the validity and uniform applicability of the result. Thus, after Wu, Ambler, Heyward and Hoppes verified the theoretically predicted non-conservation of parity in beta decay using electrons, Lederman and Steinberger verified it by studying the decay of muons (possibly aided by Garwin's suggestions). There have been at least ten different verifications of Bell's inequality, using different experimental setups and techniques, which verifies the "spooky" action-at-a-distance required by quantum measurement theory. As soon as Mossbauer announced his effect, physicists rushed to duplicate it around the world. The speed of light has been measured many, many times, as has the dilation of time predicted by the theory of special relativity. On the other hand, in medical research, there is only one direct way to do the experiment, since no other "experimental equipment" exists. This has at least two consequences:(1) there is less glory in verifying a medical result, even if it is done to higher probability by studying more people , and (2) if a result is "very convincing", then virtually no one will repeat it because it would seem to be a waste of money to the funding office, or a risk of malpractice to the research group.
I should also mention that some experiments are not done, because they are deemed not to be in the public interest:There have been several studies in Europe (usually published in Lancet) that seem to indicate that cigarette smokers have a lower incidence of Parkinson's Disease. This suggests a relationship between nicotinic receptors in the brain and dopaminergic neurons. But I predict that no one in the USA would receive federal funding to do a prospective study to see if, in fact, cigarette smoking does protect against Parkinson's Disease, or, indeed has any other benefit.
All studies of new drugs, and many studies of existing drugs, are done on pharmacologically naive patients, who are on no drugs at the time of the experiments. Since most of my patients are on at least four drugs, the results of the study may not apply to them, both as regards to benefits and side effects.(This is typified by the fact that if I have a patient with diabetes, hypertension, osteoarthritis, chronic hepatitis, and GERD, and try to follow all five of the government guidelines, drugs used to treat one problem conflict with the guidelines for another problem.)
Now that I have expressed several of my opinions, let me complete this article by reviewing definitions of several common words and phrases that you may see in research articles:
NNT---Number Needed to Treat---the statistically suggested number of patients to treat with the studied drug in order to achieve the expected outcome in one.
NNH---Number Needed to Harm---similar in concept to NNT,except it is the number to treat to get a bad outcome. If NNT is greater than NNH, you have a problem, unless, perhaps, the outcome in NNT is preventing certain death.
NNS---Number Needed to Sue----this is not generally listed in statistical textbooks. It is the number of patients out of a million who get a bad enough result that a malpractice lawyer thinks it worthwhile to start a class action suit.
Statistically significant---there is less than a 5% probability (one-in-twenty) that the result is due to chance. Alternatively, if you repeat the experiment 20 times, then you would expect to get the same "result" nineteen times.
Type I/Alpha Error---you conclude that there is a statistically significant difference between the control group and the treated group, when there is really NO difference. Similar in concept to a false positive conclusion.
Type II/Beta Error---you conclude that there is no statistically significant difference between the control and the treated group, when there really is. Similar in concept to a false negative conclusion.
Sensitivity---the probability that if you test positive for a disease, you have the disease; i.e. a test with a low false negative rate.
Specificity---the probability that if you test negative for a disease, you do not have the disease; i.e. a test with a low false positive rate.
Common clinical sense---tells you not to believe a positive test result in a particular patient. If you order a panel of 20 tests, each of which has a Gaussian distribution, then the odds are 50:50 that at least one of the tests will fall outside the "normal" range without indicating true disease.
Correlation/Causation---two events can be related in time or space without having a cause-and-effect relationship. Propinquity can always be coincidental, but can also suggest paths for future research.
Confounding---a factor not considered when looking for a cause-and-effect relationship that affects the effect. The best example would be the initial statistical demonstration that coffee drinkers had a higher rate of heart attacks, without allowing for the confounding effect that more coffee drinkers than non-drinkers smoked cigarettes. It is probably impossible to ensure the absence of all confounding effects, since we don't know about many confounding effects, and it is virtually impossible to test for their existence.
Confidence Interval---Similar in concept to error bars around a measured data point. The confidence interval of a result suggests to you the range of the result in which we expect 95% of the studied population to fall.
Intention-to-treat---included all patients who registered for the randomized drug study, whether or not they dropped out of the study.
Incidence---the percentage of the population that develops a given disease in a given period of time.
Prevalence---the percentage of the population that has the disease at a given time. Note that the prevalence of a disease helps to determine whether you want to emphasize avoiding a Type I error or emphasize avoiding a Type II error, as well as a test with high sensitivity or high specificity.
Endpoint---the result you are looking for to determine that a treatment "works".
First, it seems to be a common habit to publish data points in a graph without error bars. This makes it impossible to interpret the results properly. (For lay persons, the error bars show the range of +/- 2 standard deviations, which means that IF the data and error distribution is Gaussian, then there is a 2.5% probability that a repeat measurement would be above and a 2.5% probability of being below the range of the error bars.) Then this mistake is often compounded by connecting the data points by a sequential series of straight lines, rather than a French curve or a least squares fit.
When a result is presented as being "statistically significant", what is meant is that there is only a 5% probability of the result being incorrect (yes, I know I am simplifying here). However, a statistically significant result may not be clinically significant. It is easy to demonstrate that if you have enough subjects, something will be statistically significant. But is it really medically useful to know how to decrease your risk of being killed by a falling meteorite by 50%?
The lay press is also woefully ignorant of the concept of statistical variation. If you tabulate, for instance, cases of breast cancer in every county in a state, one county has to be the highest, and one has to be the lowest, without any "cause". Every time there is a clustering of cases (as in lymphoma in Passaic, N.J. about 15 years ago), there is a rush to find the cause.
Then the relative risk rather than absolute risk is emphasized. Again, if your chance by being killed by a falling meteorite is one in a million, then if I decrease your relative risk by 50%, I have only lowered your absolute risk by 0.0001%.
Finally, and this is the most egregious mistake of all, is the use of surrogate endpoints. For instance, in the study of the effect of lowering cholesterol by the use of Zetia, instead of looking at heart attacks or stroke as a primary endpoint, the thickness of the intima of the carotid artery was used as a surrogate endpoint. If the most common cause of arterial blockage is rupture of a plaque rather than embolic, then this surrogate endpoint is not medically useful. (Personally, I think the evidence points to the anti-inflammatory effects of ASA and statins as reducing the risk of plaque rupture and acute blockage, but no one has yet been able to detect such an acute event when it happens in humans.)
Actually, in fairness to medical researchers, I should mention the main limitation that they face. In physics, if an experimental result is published, other researchers rush to try to repeat the result by a different experimental technique, using different apparatus, to help establish the validity and uniform applicability of the result. Thus, after Wu, Ambler, Heyward and Hoppes verified the theoretically predicted non-conservation of parity in beta decay using electrons, Lederman and Steinberger verified it by studying the decay of muons (possibly aided by Garwin's suggestions). There have been at least ten different verifications of Bell's inequality, using different experimental setups and techniques, which verifies the "spooky" action-at-a-distance required by quantum measurement theory. As soon as Mossbauer announced his effect, physicists rushed to duplicate it around the world. The speed of light has been measured many, many times, as has the dilation of time predicted by the theory of special relativity. On the other hand, in medical research, there is only one direct way to do the experiment, since no other "experimental equipment" exists. This has at least two consequences:(1) there is less glory in verifying a medical result, even if it is done to higher probability by studying more people , and (2) if a result is "very convincing", then virtually no one will repeat it because it would seem to be a waste of money to the funding office, or a risk of malpractice to the research group.
I should also mention that some experiments are not done, because they are deemed not to be in the public interest:There have been several studies in Europe (usually published in Lancet) that seem to indicate that cigarette smokers have a lower incidence of Parkinson's Disease. This suggests a relationship between nicotinic receptors in the brain and dopaminergic neurons. But I predict that no one in the USA would receive federal funding to do a prospective study to see if, in fact, cigarette smoking does protect against Parkinson's Disease, or, indeed has any other benefit.
All studies of new drugs, and many studies of existing drugs, are done on pharmacologically naive patients, who are on no drugs at the time of the experiments. Since most of my patients are on at least four drugs, the results of the study may not apply to them, both as regards to benefits and side effects.(This is typified by the fact that if I have a patient with diabetes, hypertension, osteoarthritis, chronic hepatitis, and GERD, and try to follow all five of the government guidelines, drugs used to treat one problem conflict with the guidelines for another problem.)
Now that I have expressed several of my opinions, let me complete this article by reviewing definitions of several common words and phrases that you may see in research articles:
NNT---Number Needed to Treat---the statistically suggested number of patients to treat with the studied drug in order to achieve the expected outcome in one.
NNH---Number Needed to Harm---similar in concept to NNT,except it is the number to treat to get a bad outcome. If NNT is greater than NNH, you have a problem, unless, perhaps, the outcome in NNT is preventing certain death.
NNS---Number Needed to Sue----this is not generally listed in statistical textbooks. It is the number of patients out of a million who get a bad enough result that a malpractice lawyer thinks it worthwhile to start a class action suit.
Statistically significant---there is less than a 5% probability (one-in-twenty) that the result is due to chance. Alternatively, if you repeat the experiment 20 times, then you would expect to get the same "result" nineteen times.
Type I/Alpha Error---you conclude that there is a statistically significant difference between the control group and the treated group, when there is really NO difference. Similar in concept to a false positive conclusion.
Type II/Beta Error---you conclude that there is no statistically significant difference between the control and the treated group, when there really is. Similar in concept to a false negative conclusion.
Sensitivity---the probability that if you test positive for a disease, you have the disease; i.e. a test with a low false negative rate.
Specificity---the probability that if you test negative for a disease, you do not have the disease; i.e. a test with a low false positive rate.
Common clinical sense---tells you not to believe a positive test result in a particular patient. If you order a panel of 20 tests, each of which has a Gaussian distribution, then the odds are 50:50 that at least one of the tests will fall outside the "normal" range without indicating true disease.
Correlation/Causation---two events can be related in time or space without having a cause-and-effect relationship. Propinquity can always be coincidental, but can also suggest paths for future research.
Confounding---a factor not considered when looking for a cause-and-effect relationship that affects the effect. The best example would be the initial statistical demonstration that coffee drinkers had a higher rate of heart attacks, without allowing for the confounding effect that more coffee drinkers than non-drinkers smoked cigarettes. It is probably impossible to ensure the absence of all confounding effects, since we don't know about many confounding effects, and it is virtually impossible to test for their existence.
Confidence Interval---Similar in concept to error bars around a measured data point. The confidence interval of a result suggests to you the range of the result in which we expect 95% of the studied population to fall.
Intention-to-treat---included all patients who registered for the randomized drug study, whether or not they dropped out of the study.
Incidence---the percentage of the population that develops a given disease in a given period of time.
Prevalence---the percentage of the population that has the disease at a given time. Note that the prevalence of a disease helps to determine whether you want to emphasize avoiding a Type I error or emphasize avoiding a Type II error, as well as a test with high sensitivity or high specificity.
Endpoint---the result you are looking for to determine that a treatment "works".
Subscribe to:
Posts (Atom)