On 8/15/97 3:19 AM, Jonathan Siegel,
JMSiegel.Info.Research@worldnet.att.net said:
Please excuse the cross posting of this somewhat long message from the DEN
list, but it is a wonderful summary of what has been wrong with our
*discussion* of performance management from Deming's perspective. I
thought you'd enjoy the usefulness of the underlying thoughts of the
posting, both as it affects this discussion, as well as others on the Net.
-- Jack Zigon
>I have listened with interest to the discussion on whether Deming was
>wrong in his view that performance appraisals, particularly formal ones,
>are always more harmful than beneficial.
>
>I am interested in the way that debates about Dr. Deming's ideas are
>discussed. One question I have often asked is, are Dr. Deming's ideas
>theories in the scientific sense, capable of being established or
>refuted (to some degree of risk of being wrong) by evidence?
>
>The majority of the discussion seems to have taken four forms:
>repetition, authority, personal experience, and morality.
>
>The argument from repetition consists of repeating Dr. Deming's
>arguments in various forms, without substantive evidence. It is almost
>as if shear force of rhetoric could carry the day.
>
>The argument from authority consists of citations to Dr Deming or his
>colleagues/students. This authority is offered as proof of the truth of
>what is asserted.
>
>The argument from experience -- by far the most common argument --
>consists of personal anecdotes which in the author's view support the
>claim. Even if the individuals' assessment of the causes of their life
>experience is accurate, no number of isolated anecdotes can establish a
>general theory. And refuting its opposite is not helpful, since nobody
>has argued that performance appraisal is always good.
>
>The argument from morality consists of the view that performance
>evaluations are in and of themselves morally wrong and sinful. Such an
>argument cannot generally be verified scientifically. Bertrand Russell
>has summed up thousands of years of human moral thought by saying that
>even though he has come up with no way of proving why his dislike of
>torture is not as much a matter of pure personal taste as his dislike
>of, say, broccolli, yet he could not for a minute bring himself to take
>such an equivalence seriously. Because our attempts to express moral
>propositions in scientific or even merely rigorous form never seem to do
>justice to them, purely moral arguments seem to be unavoidable.
>
>Yet none of these four arguments -- repetition, authority, experience,
>or morality -- are sufficient to establish or refute a theory. By
>relying on these arguments, the overwhelming majority of the DEN
>speakers addressing the subject is implicitly rejecting Dr. Deming's
>assertions that "theory" is the appropriate term to describe his
>management ideas.
>
>If Dr. Deming's ideas are a theory, than we should be able (at least
>conceptually, however hard in practice) to gather a body of evidence
>that tends to show that performance appraisal consistently causes the
>kind of losses Deming claimed, and/or do not provide the benefits
>claimed by its supporters. Anton Tolman has noted that nearly all the
>scientific evidence amassed in the studies cited by Alfie Kohn and
>others was gathered in non-organizational settings, and is not
>necessarily valid in the context of the workplace. I don't know enough
>about the field to know if this is so or not.
>
>But I find myself puzzled that we are discussing the issue the way we
>are. Why aren't we asking what the evidence is? Why aren't we discussing
>ways to acquire new evidence or close holes in the existing evidence?
>Why does systematic evidence seem to be absent from, or at most, at the
>periphery of this discussion rather than at its center? Scientific
>inquiry seems to be the last thing that would occur to us as a way to
>resolve disputes of this sort. Yet from everything I understand about
>Dr. Deming's work, this is at the center of his ideas about how to
>succeed.
>
>Without a systematic, rational way to address disputes, how can we hope
>for debate ever to be anything more than a shouting match? And why
>should we expect the management of a company to be based on anything but
>politics or, at best, wishful thinking?
>
>I would urge everyone to review the posts on this topic (and others) and
>look carefully about the kinds of arguments being used. Each of the four
>kinds of arguments (repetition, authority, experience, and morality) has
>some place in exploring issues. But I believe much of the Deming
>philosophy stands for the proposition that there is a better way for
>members of an organization to discuss what works and how to proceed.
>
>Jonathan Siegel
What a refreshing post!
I had just about given up on the DEN list as it seemed that only
repetition and authority were important. I understood Deming to say that
ideas aren't right/wrong but useful/not useful based on the data. But not
all "followers" are as broad minded. (And I use the term "followers" as
tongue-in-cheek ;-) )
Only one person has asked me for any *data* on performance appraisal's
affect on employee performance. Most offered personal experience of being
shafted by a poorly designed system being misused by an incompetent and
possibly malevelent supervisor.
To counter this anecdotal data, in 1981, I designed, implemented and
evaluated the effects of a performance appraisal system on a large US
trucking company. The effects of systematically defining expectations,
providing ongoing feedback, coaching and incentives produced $19.8
million in performance improvements during the first 6 months based on
data from about 1/3 of the 600 locations affected. The cost of the
program was approximately $1 million, so we had a 20:1 ROI
(conservatively). Data collection was stopped by upper management once
the effectiveness of the program had been established.
This case study was published most recently in "Measuring Return On
Investment, Vol. 1", Jack Phillips, Ed,. American Society for Training
and Development, 1994 (703/683-8100). I also have the article
(YFSSAVE.ZIP) available to email in Word for Windows format to those
interested. It's not perfect research, but an attempt to find out what
works and what doesn't.
One interesting negative finding was that only 88% of the managers ended
up using the system as designed; 12% did not use it. We worked hard to
see if there were *system* reasons for the failure -- but sometimes the
problem was the supervisor. He just didn't want to do the work required
of the supervisory role.
Based on this successful implementation, I've refined the
measurement/feedback process, especially focusing on ways to measure the
really hard-to-measure work in our society: R&D scientists, design
engineers, graphic artists, health care workers and most recently teams
made of these kinds of workers.
I've documented the process in articles, presentations and how-to books
for others to read, critique, learn from, and apply to their situations.
I offer articles describing the process for white-collar work
(WHTCOLLR.ZIP) and team results measurement (SEVENSTP.ZIP) to anyone
interested in critiquing the process and seeing if it can add to the body
of knowledge.
Those who are "religious" believers won't want to be confused by facts
which don't fit their "faith". (And you know who you are... :-)...) Those
who are seekers of knowledge and see their body of theory as one which
can grow and incorporate useful information from other disciplines may
find something of use here.
My hunch is that even true believers are lacking ways to measure *all*
the kinds of work they are faced with. Saying "the most important things
in life is unknowable" is a cop out. Without the courage to venture into
the unknown, and the belief that it can be done, we'd all still be
convinced that the world is flat.
I'm all for trying to use data to improve systems and their outputs --
and not blame employees for the failings of poorly designed and managed
systems. If you are satisfied with your ability to measure performance,
this post is a waste of your time. If you see a need to improve
technology in this area, perhaps we have something to teach each other.
I for one have learned about how to go about discussing issues like this
from Jonathan Siegel. Thanks!
-- Jack Zigon President Zigon Performance Group PO Box 520 Wallingford, PA 19086-0520Specializing in Performance Management for teams and hard-to-measure jobs
Email: jack@zigonperf.com URL: www.zigonperf.com
Voice: 610-891-9599 Fax: 610-891-9055 Orders: 800-244-2892 Fax-on-demand: 800-299-3022
Learning-org -- An Internet Dialog on Learning Organizations For info: <rkarash@karash.com> -or- <http://world.std.com/~lo/>