The Seven Warning Signs of Bogus Science
The Chronicle of Higher Education, 3.1.31
By ROBERT L. PARK
The National Aeronautics and Space Administration is
close to a million dollars in an obscure Russian
scientist's antigravity machine, although it has failed every
test and would violate the most fundamental laws of nature. The Patent and Trademark Office recently
issued Patent 6,362,718 for a physically impossible motionless
electromagnetic generator, which is supposed
to snatch free
energy from a vacuum. And major power companies have sunk tens of millions of dollars
into a scheme to produce energy by putting hydrogen atoms into a state below their ground state, a feat
equivalent to mounting an expedition to explore the region south of the South Pole.
There is, alas,
no scientific claim so preposterous that a
scientist cannot be found to vouch for it. And many such
end up in a court of law after they have cost some
gullible person or corporation a lot of money. How are juries
to evaluate them?
Before 1993, court cases that hinged on the validity of
claims were usually decided simply by which expert witness the jury found more credible. Expert testimony often consisted
of tortured theoretical speculation with little or no supporting evidence. Jurors were bamboozled by technical gibberish
they could not hope to follow, delivered by experts whose credentials they could not evaluate.
In 1993, however, with the Supreme Court's landmark decision in Daubert v. Merrell Dow Pharmaceuticals,
Inc. the situation began to change. The case involved Bendectin, the only morning-sickness medication ever
approved by the Food and Drug Administration. It had been used by millions of women, and more than 30 published
studies had found no evidence that it caused birth defects. Yet eight so-called experts were willing to
testify, in exchange for a fee from the Daubert family, that Bendectin might indeed cause birth defects.
In ruling that such testimony was
not credible because of lack of supporting evidence, the court instructed federal judges to serve as "gatekeepers," screening
juries from testimony based on scientific nonsense. Recognizing that judges are not scientists, the court
invited judges to experiment with ways to fulfill their gatekeeper responsibility.
Justice Stephen G. Breyer encouraged trial judges to appoint independent
experts to help them. He noted that courts can turn to scientific organizations, like the National Academy of Sciences
and the American Association for the Advancement of Science, to identify neutral experts who could preview questionable
scientific testimony and advise a judge on whether a jury should be exposed to it. Judges are still concerned
about meeting their responsibilities under the Daubert decision, and a group of them asked me how to recognize
questionable scientific claims. What are the warning signs?
I have identified
seven indicators that a scientific claim
lies well outside the bounds of rational scientific discourse. Of
course, they are only warning signs -- even a claim with several of the signs could be legitimate.
1. The discoverer pitches the claim directly to the media. The integrity
of science rests on the willingness of scientists to expose new ideas and findings to the scrutiny of other scientists.
Thus, scientists expect their colleagues to reveal new findings to them initially. An attempt to bypass peer review
by taking a new result directly to the media, and thence to the public, suggests that the work is unlikely to stand
up to close examination by other scientists.
One notorious example is the claim made in 1989 by two
from the University of Utah, B. Stanley Pons and
Martin Fleischmann, that they had discovered cold fusion -- a
way to produce nuclear fusion without expensive equipment. Scientists did not learn of the claim until
they read reports of a news conference. Moreover, the announcement dealt largely with the economic potential
of the discovery and was devoid of the sort of details that might have enabled other scientists to judge
the strength of the claim or to repeat the experiment. (Ian Wilmut's announcement that he had successfully
cloned a sheep was just as public as Pons and Fleischmann's claim, but in the case of cloning, abundant scientific
details allowed scientists to judge the work's validity.)
Some scientific claims avoid even the scrutiny
of reporters by appearing in paid commercial advertisements. A health-food company marketed a dietary supplement
called Vitamin O in full-page newspaper ads. Vitamin O turned out to be ordinary saltwater.
2. The discoverer says that a powerful establishment is trying to suppress
his or her work. The idea is that the
establishment will presumably stop at nothing to suppress
that might shift the balance of wealth and power in society. Often, the discoverer describes mainstream science as
part of a larger conspiracy that includes industry and government. Claims that the oil companies are frustrating
the invention of an automobile that runs on water, for instance, are a sure sign that the idea of such a
car is baloney. In the case of cold fusion, Pons and Fleischmann blamed their cold reception on physicists
who were protecting their own research in hot fusion.
3. The scientific effect involved
is always at the very limit of detection. Alas, there is never a clear photograph of a flying saucer, or
the Loch Ness monster. All scientific measurements must contend with some level of background noise or statistical
fluctuation. But if the signal-to-noise ratio cannot be improved, even in principle, the effect is probably not
real and the work is not science.
Thousands of published papers in para-psychology, for example, claim to report verified instances
psychokinesis, or precognition. But those effects show up only in tortured analyses of
statistics. The researchers can find no way to boost the signal, which suggests that it isn't really there.
Evidence for a discovery is anecdotal. If modern science
has learned anything in the past century, it is to distrust
evidence. Because anecdotes have a very strong
emotional impact, they serve to keep superstitious beliefs
in an age of science. The most important discovery of
modern medicine is not vaccines or antibiotics, it is the
double-blind test, by means of which we know what works and what doesn't. Contrary to the saying, "data" is not the
plural of "anecdote."
5. The discoverer says a belief
is credible because it has
endured for centuries. There is a persistent myth that
hundreds or even
thousands of years ago, long before anyone
knew that blood circulates throughout the body, or that germs cause
disease, our ancestors possessed miraculous remedies that modern science cannot understand. Much of what is termed
"alternative medicine" is part of that myth.
Ancient folk wisdom, rediscovered or repackaged, is
unlikely to match the output of modern scientific laboratories.
6. The discoverer has worked in isolation. The image of a lone genius who struggles in secrecy
in an attic laboratory and ends up making a revolutionary breakthrough is a staple of Hollywood's science-fiction
films, but it is hard to find examples in real life. Scientific breakthroughs nowadays are almost always
syntheses of the work of many scientists.
7. The discoverer must propose new laws of nature to explain an
observation. A new law of nature, invoked to explain some extraordinary result, must not conflict with what is already
known. If we must change existing laws of nature or propose new laws to account for an observation, it is
almost certainly wrong.
I began this list of warning signs to help federal judges
scientific nonsense. But as I finished the list, I
realized that in our increasingly technological society,
voodoo science is a skill that every citizen should develop.
Robert L. Park is a professor of physics
at the University of Maryland at College Park and the director of public
information for the American
Physical Society. He is the
author of Voodoo Science: The Road From Foolishness to Fraud, Oxford University