E-Learning Curve Blog

Using the Program Logic Model to evaluate e-learning

Returning to the topic of undertaking a case study when evaluating any e-learning initiative, we can say that it is essential that the data collected in the research is

• i. relevant

and

• ii. appropriate

to the task of evaluating the impact of the intervention on the audience.

According to Wright & Ross (2001), the Program Logic Model

offers a conceptual and practical framework for this exacting work. At its core it is a simplified picture of a programme (sic), initiative or project. In much the same way that a software developer would employ a use case, the logic model shows the relationships among the resources that are invested, the intervention or the activities that take place, and the outcomes that result from the project. Essentially, it is a tool to guide critical thinking.

(p.153)

Figure 1 The Program Logic Model

This model offers great utility to the e-learning practitioner, specifically in orienting the research and ensuring their learning evaluation methodology aligns to the investigation being undertaken. As Cooksy, Gill and Kelly discuss in their article The Program Logic Model as an Integrative Framework for a Multimethod Evaluation,

…program theory guides an evaluation by identifying key program elements and articulating how these elements are expected to relate to each other.

(p.109)

As illustrated in Figure 1 above, there are five components to the Program Logic Model: each component has a range of characteristics associated with it. These are described by Paul McCawley (1997):

1. The situation statement provides an opportunity to communicate the relevance of the project. This establishes a baseline for comparison at the close of a project and provides a way to determine whether change has occurred. Describing who is affected by the initiative allows assessment of who has benefitted.
2. Inputs include those things that we invest in a program or project such as knowledge, skills, or expertise. Describing the inputs needed for a project provides an opportunity to communicate the quality of the program.
3. Outputs are “those things that we do” (p.4) – providing deliverables, products – and the people we reach – intended audience. Describing the outputs establishes linkages between the problem (situation) and the impact of the project (intended outcomes).
4. In the context of program or project outcomes, these can be short-term, intermediate-term, or long-term, Outcomes answer the question “What happened as a result of the program?”

• Short-term outcomes of educational programs may include changes in:
  • Awareness – customers recognize the problem or issue
  • Knowledge – customers understand the causes and potential solutions
  • Skills – customers possess the skills needed to resolve the situation
  • Motivation – customers have the desire to effect change
• Intermediate-term outcomes include changes that follow the short-term outcomes, such as changes in:
  • Practices used by participants
  • Behaviors exhibited by people or organizations
• Long-term outcomes follow intermediate-term outcomes when changed behaviors result in changed conditions or processes.

More Next Time…

_________

References:

Cooksy, L. J. Gill, P. & Kelly. P. A. (2001) The Program Logic Model as an Integrative Framework for a Multimethod Evaluation. Evaluation and Program Planning [Internet] 24(2). Available from: http://www.hsrd.houston.med.va.gov/AdamKelly/Logic.html [Accessed 21st October, 2006]

McCawley, P. F. (1997) The Logic Model for Program Planning and Evaluation. University of Idaho Extension [Internet] Available from: http://www.uidaho.edu/extension/LogicModel.pdf [Accessed 18th January 2007]

Wright, L. & Ross, J. W. (2001) The Logic Model: An A-to-Z Guide to Training Development, Implementation, and Evaluation. Association of the American Public Human Services Association (APHSA). [Internet] Available from: http://calswec.berkeley.edu/CalSWEC/2001_08Ross.pdf [Accessed 10th January 2007]

–

July 06 2010 03:00 pm | e-learning

One Response to “Using the Program Logic Model to evaluate e-learning”

1. Reading on the Interwebs « IndigoWorks on 15 Jul 2010 at 7:23 pm #

   [...] Evaluating e-learning. This is a great, high-level model for measuring the effectiveness of an e-learning program. I like this as a starting point but I think any training organization should think about how each step is implemented for them. Define those and they have some actions they can take. But this model would be great for any team to start their e-learning evaluation process. [...]