Replying to LO25269 --
Dear Organlearners,
Richard Karash <Richard@Karash.com> writes:
>When I first made the career change from high-tech
>management to organizational learning consultant,
>I was taught by my clients this four level scheme for
>thinking about evaluation of training and development:
Greetings Rick,
I am not fond of the term "training" -- it has too many connotations of
forcing animals into non-spontaneous behaviour for me. But if I have to
use it because it is the term which others prefer to use, I will do it.
Unfortunately, this will lead to rather curious descriptions such as
"spontaneous training" and "double loop training".
Any way, the four levels to which you refer to, signifies a "systematics
of learning" as I have explained in another contribution (The Many Faces
of Variety) which I have mailed some minutes ago.
>Here's what I take from the above four level consideration:
>We're really interested in level 4, but most training evals
>are level 1. If we're serious about evaluation of T&D, it's a
>big job, not an add-on effort.
Yes, it is definitely a complex job. Compare your "4 level systematics" to
"six level systematics" of Bloom et al which I have mentioned in The Many
Faces of Variety. In each systematics there is an increase of complexity
from the one in it to the next level.
>p.s. About the "narrow definition of learning"... I like to
>think of knowledge as the capacity for effective action.
>Then, learning is an increase in the capacity for effective
>action, and might not be well measured by conventional
>subject matter tests.
You are right, but why? The first question we have to ask ourselves is if
it is possible to formulate the effective action in each level by means of
learning objectives.
After thirty years I am pretty much convinced that it is possible. What
we can speak of in terms of declarative sentences we can also speak of in
terms of imperative sentences. The only problem is that we may have not
yet developed our capacity for using imperatives.
The second question is if a test item and a learning objective is the same
thing? No, a test item is a model which we design so as to explore the
mastery of a learning objective. If a learning objective belongs to a
higher level of complexity in the systematics of learning, it is to be
expected that conventional test items for it may not yet have been
designed. (Complexer things take longer time to get created.) Creating
models for testing such complex learning objectives is, using your words,
a "big job".
I would strongly suggest to fellow learners to first establish a
systematics for the T&D which they have in mind before going to the
evaluating stage.
I would also warn them from experience that this "systematics" is
intimately connected to the content of what has to be trained. In other
words, the systematics of one T&D program cannot be copied to become
unchanged the systematics of any other T&D program. Each T&D program has
its own authentic systematics which will have to be set up by hard work.
Systematic by recipe is a recipe for failure.
In biology, for example, one soon learns that the systematics of animals
is different to the systematics of plants, or that the systematics of
monocolytodons are different to that of the dicotylodon plants. Yet for
some curious reason, this insight lacks in T&D. As an example I may offer
Curriculum 2000. It was a project formulated after the fall of apartheid
to open up the education to all the peoples of South Africa. It worked
with "learning outcomes". But it has become gradually geopardised
One main reason was the extreme rigidity in which the overall systematics
was forced upon every subject and every level of schooling. There is a
reason for this. Somehow many people had the Mental Model that modifying
the systematics for the particular content to be learned and the level at
which it had to be learned, was a clever return to the discrimination of
apartheid.
With care and best wishes
--At de Lange <amdelange@gold.up.ac.za> Snailmail: A M de Lange Gold Fields Computer Centre Faculty of Science - University of Pretoria Pretoria 0001 - Rep of South Africa
Learning-org -- Hosted by Rick Karash <Richard@Karash.com> Public Dialog on Learning Organizations -- <http://www.learning-org.com>
"Learning-org" and the format of our message identifiers (LO1234, etc.) are trademarks of Richard Karash.