Oh my, we aren't really still having this argument are we?
One of our colleagues writes (and I won't single him out because I take
him to be representative of a way of thinking in many social science
fields):
>You will -never- be able to determine how much extra business that
>excellent customer service might have brought you, or how much merely
>middling customer service might have cost you.
That is really odd, since that's what I've been doing for the past dozen
years -- with quite a lot of success, actually!
I would invite anyone contemplating issues of measurement to remember
something very simple and fundamental, viz:
"Anything that exists, exists in some quantity."
As to whether it can be accurately enough measured, I would suggest
further that we stop thinking about measurements like the kind we make
with a ruler and think about other ways we measure things. Even in the
measurement of distance I could hear social scientists making absurd
comments such as, "You can't possibly measure the distance to the stars
because no one can make a ruler that is several light years long!"
Ridiculous. That's confusing the impossibility of doing something in one
particular way with the impossibility of doing something at all.
All measurement, even with a ruler, struggles with the issue of confidence
in the quantity measured. Some physical measurements have errors so small
as to be ignorable in the context the measurement is to be used. To
address this confidence issue, most successful social science measurements
that I'm aware of use some variant of the "rule of triangulation." That
is, knowing that it is difficult to get a single metric that confidently
captures the essence of what we want to measure, we find a way to provide
"evidence" about the quantity of something in three or four ways. If all
of the evidence points to the same quantity (or whatever is "close enough"
in the context of use), we feel quite confident that we know that
quantity.
As a final point, you'll notice that I slipped a word in the previous
paragraph that might have been unexpected -- "evidence." I have found
that people are more comfortable (because they cannot stop thinking about
rulers) in talking about creating "evidence" about the quantity of
something like "learning" rather than in creating a "measure." I would
argue that, by triangulation, if you put together three (or more) ways of
generating "evidence" you have a composite "measure" that can be as
accurate as you need.
But, if you don't buy that, at least we could change the dialogue and talk
about "evidence" of learning and how to generate it credibly.
--"John W. Gunkler" <jgunkler@sprintmail.com>
Learning-org -- Hosted by Rick Karash <rkarash@karash.com> Public Dialog on Learning Organizations -- <http://www.learning-org.com>