Replying to LO25219 --
When I first made the career change from high-tech management to
organizational learning consultant, I was taught by my clients this four
level scheme for thinking about evaluation of training and development:
Level 1: Did the participants like it? (e.g., measured by "smile sheets"}
Level 2: Did they learn what they were supposed to learn? (e.g., as
measured by tests of the subject matter, like we all had in school. Notice
the implied definition of learning is narrower than we'd like in the org
learning field.)
Level 3: Can they do it in context? (e.g., as measured by examining
actions and behaviors in the workplace after the training.)
Level 4: Does it make a difference? (In theory, we could conduct
test/control comparisons to see if the actions and behaviors, when done,
make a difference. In practice, this is very difficult.)
Here's what I take from the above four level consideration: We're really
interested in level 4, but most training evals are level 1. If we're
serious about evaluation of T&D, it's a big job, not an add-on effort.
One of my clients, with a world-wide reputation as a leader in T&D, set an
objective a couple years ago to conduct good Level 2 evals of all their
training activities.
-=- Rick
p.s. About the "narrow definition of learning"... I like to think of
knowledge as the capacity for effective action. Then, learning is an
increase in the capacity for effective action, and might not be well
measured by conventional subject matter tests.
p.p.s. I believe the four levels above are something of a classic
piece in T&D literature, but I don't know whom to credit.
-=- Rick
>I am currently doing an assignment on Training and Development. Does
>anyone have any suggestions or ideas the types of outcomes I would
>recommend to use in evaluating on training program and which evaluation
>design I would use? This design will be based on minimizing threats to
>validity and practical considerations.
>
>"JJ" <ehnas@hotmail.com>
--Richard Karash ("Rick") | <http://world.std.com/~rkarash> Speaker, Facilitator, Trainer | mailto:Richard@Karash.com "Towards learning organizations" | Host for Learning-Org Discussion (617)227-0106, fax (617)523-3839 | <http://www.learning-org.com>
Learning-org -- Hosted by Rick Karash <Richard@Karash.com> Public Dialog on Learning Organizations -- <http://www.learning-org.com>
"Learning-org" and the format of our message identifiers (LO1234, etc.) are trademarks of Richard Karash.