Ouch! Leo, you have put very plainly some of my fears about what has
happened in science in my lifetime. Thanks (I think) for describing the
phenomenon so well.
It is very difficult to get one's hands on "raw data" anymore, isn't it?
And, what's worse, even if one could get at it could one trust it? Let me
describe a situation I ran into as a first-year graduate student.
My assigned advisor had a worldwide reputation for having published some
research on whether the addition of images/graphics to text made learning
to read easier or harder. His paper continues to be cited 100's of times
a year.
He decided he wanted to do a follow-up study and asked me to get out the
original data from a file he had kept. I did -- and was horrified to
discover that he had not reported all the data he had collected! [Only
the confirmatory data, of course.] After a heated discussion where he
tried to justify this dishonesty, I threatened to report him, then left
and asked to be given a different advisor -- and never spoke to him again.
I am afraid that, with the pressure to publish and the unwillingness of
journal editors (as Leo says) to publish observations without some new
"theory" that comes from it -- and the longstanding unwillingness of
journals to publish negative results -- my former advisor is not the only
researcher who is dishonest.
What can we do? I fear that the entire scientific community, and its
culture, must be changed to correct this situation and I don't think I
have the will to take it on. It's things like this that have made my
"crap detector" such a well-used implement, and I don't like to be so
cynical -- but I also don't want to build edifices of thought and practice
on the "proof" of Dr. Jay Samuels that pictures enhance/don't enhance
learning to read.
--"John Gunkler" <jgunkler@sprintmail.com>
Learning-org -- Hosted by Rick Karash <rkarash@karash.com> Public Dialog on Learning Organizations -- <http://www.learning-org.com>