Think about the following the next time you read a medical journal.
“Rethinking the Admissibility of Medical Treatises as Evidence” . JP Lipton, M O’Connor, BD Sales. American J Law & Medicine 1991 17:209-248
“…there are scarcely any bars to eventual publication. There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too justified, and no grammar and syntax too offensive for a paper to end up in print.”
Sheps & Schecter: The assessment of diagnostic tests: a survey of current medical research. JAMA 1984, 2418:2420-21. This paper found a great potential for plagiarism and fraud. I’m sure there are many other examples, these are just the first two that come to mind for me.
The moral of the story? Read carefully and take the time to see if the article is really credible and, also importantly, passes the “so what?” test. Was there a sufficient sample size? Were the appropriate methodologies used? Where the correct statistical tests used? Do they support the conclusion? Does the conclusion matter, i.e. is it adding something important to the literature? If the conclusion seems suspicious, does the author have a history of publishing research in the field supporting the theory or method of thinking?
Follow the above questions each time you read a journal article and you may be amazed at some of the things that stand out in the literature.
As a side note, another good way to look at scientific claims is the 10 questions of the Baloney Detection Kit from Michael Shermer which are:
1. How reliable is the source of the claim?
2. Does the source make similar claims?
3. Have the claims been verified by somebody else?
4. Does this fit with the way the world works?
5. Has anyone tried to disprove the claim?
6. Where does the preponderance of the evidence point?
7. Is the claimant playing by the rules of science?
8. Is the claimant providing positive evidence?
9. Does the new theory account for as many phenomena as the old theory?
10. Are personal beliefs driving the claim?
Filed under: Medicine | Leave a comment »
The Voice of Reason 