Can We Align Kirkpatrick’s Guidelines & NonFormal Learning?
In my previous post on this topic, I outlined Kirkpatrick’s guidelines for evaluating training programs (see Table 1).
Table 1 Guidelines for evaluating learning
|
Guidelines for evaluating learning |
|
1. Use a control group if practical |
|
2. Evaluate knowledge skills and/or attitudes both before and after the program |
|
3. Use a paper-and-pencil test to measure knowledge and attitudes |
|
4. Use a performance test to measure skills |
|
5. Get a 100 per cent response |
|
6. Use the results of the evaluation to take appropriate action |
In this post, I will discuss the difficulties encountered in applying these guidelines to non-formal learning programs.
Now read on…
Kirkpatrick’s guidelines are incompatible with the assessment of non-formal learning in three ways.
First, the singular nature of each non-formal learning event means that no control group can exist to measure a difference between a control group and the “experimental group” (p.43). Similarly; a significant number of learners that access non-formally delivered content do so asynchronously, via a number of different learning channels. We must also account for the dynamic of the delivery channel: asynchronous e-learning has a long tail. I would assert that this is one of the strengths of e-learning; but the element I want to highlight here is that learners access this content in a relatively ad hoc manner, over a long timeframe. Understanding the “just enough, just in time” aspect of e-learning (non-formal or otherwise) means we have to accept that it is unrealistic to attempt to measure learning using an experimental method.
Secondly, the lack of summative assessment in non-formal learning precludes both pencil-and-paper and performance testing to measure learning.
Finally, the distributed nature of access to these events, over both time and location makes achieving or even maintaining a 100 per cent learner response rate (expressed by the value R in the example below) is impractical, and may even be impossible to attain.
In an e-learning context, we can say that as long as the e-learning content is available, the potential always exists for the total number of learners to increment upwards. A complete learner response rate will always be a hostage to this potential for learners to increment. It follows that you can never achieve a one hundred per cent response rate.
Contrast this with formal, classroom-based training courses. These courses are, by their nature, finite: each course has a specified duration. Content is delivered to learners in an instructor-led environment in a sequence of modules over the duration of the course. Once the course instruction is complete, members of any given class are required to take a comprehensive summative assessment on the learning objectives outlined in the course, usually within a set time after they complete a course to achieve certification. For example, let’s take an ILT course of 10 modules, with 12 learners participating for 8 hours, with a mandatory summative assessment to be completed at the conclusion of the instruction time. In this case, as we know the values for number of learners, duration of course, and number of modules, we can capture a 100 per cent response rate to the course in the summative testing phase.
I believe that it is possible to capture and use data with the retrospective pre-test method, and I’ll be discussing an approach to undertaking this activity next time.
_________
References:
Kirkpatrick, D. & Kirkpatrick, P. (2006). Evaluating Training Programs (3rd ed.) San Francisco, CA: Berrett-Koehler Publishers, Inc.
–
July 20 2010 03:30 pm | e-learning
2 Responses to “Can We Align Kirkpatrick’s Guidelines & NonFormal Learning?”
Evaluating Non-Formal Learning Programs - Table of Links | E-Learning Curve Blog on 05 Aug 2010 at 3:32 pm #
[...] Can We Align Kirkpatrick’s Guidelines & Non-Formal Learning? [...]
Top 10 E-Learning Curve Blog posts for 2010 | E-Learning Curve Blog on 14 Dec 2010 at 3:04 pm #
[...] Can We Align Kirkpatrick’s Guidelines & Non-Formal Learning? [...]