E-Learning Curve Blog

Evaluating Non-Formal Learning Programs – Table of Links

Here at the E-Learning Curve Blog, all my posts end with a double em-dash or ——.

I don’t know how or why I got into the habit of including the punctuation mark in this way, but it’s pretty ingrained at this point, so get used to it: it’s going to be around for a while…

…except for today.

Today is one of those occasions when I finally conclude a topic series, so something different happens today.

More on this anon.

As you’ll know if you’ve been following my blog for the last two months, I have been discussing – at great length and in some detail – the subject of evaluating non-formal learning programs. I haven’t said every I want to say on the topic, but this blog is about all aspects of e-learning, and it’s time to discuss some other areas of the discipline. Over the next few months I’ll be looking at

• m-learning
• some interesting approaches to instructional design
• content development approaches
• the state of e-learning industry
• and a whole bunch of other, interesting material.

That’s for the future: back to the present.

To wrap up this series of posts in a neat little package, here is the complete table of links to each post in the series Evaluating Non-Formal Learning.

1. Non-Formal Learning – Evaluation Considerations
2. Measuring the Learning Effect of Non-Formal Learning
3. Evaluating Non-Formal Learning in the Context of Kirkpatrick’s Four-Level Model
4. Evaluating Non-Formal Learning using Kirkpatrick’s Four-level model: More…
5. Yet more on Evaluating Non-Formal Learning
6. Approaches to Evaluating Non-Formal Learning: the Case Study
7. Evaluating Non-formal Learning: Using ‘tests of rigor’ for validation
8. Evaluating Non-formal Learning: Validity in Research
9. Using the Program Logic Model to evaluate e-learning
10. More on the Program Logic Model and E-Learning
11. A Research Design to Evaluate E-Learning Projects
12. Case Study Techniques for Evaluating E-Learning Initiatives
13. Using a Retrospective Pretest Design in Elearning Evaluation
14. Pros and Cons of the Retrospective Pre-Test Design
15. Kirkpatrick’s Guidelines for Evaluating Training Programs
16. Can We Align Kirkpatrick’s Guidelines & Non-Formal Learning?
17. Using Quantitative Data when Evaluating Non-Formal Learning
18. Using readiness surveys when evaluating e-learning
19. Is your Organization Ready for E-Learning?

Use them wisely, my friends; it’s powerful medicine.

“So,” I hear you ask, “how are you signing off this blog post, Michael?”

Patience, impetuous youth!

If you’ve ever watched the HBO program The Wire you’ll know that each season of The Wire focuses on a different facet of the city of Baltimore, MD. They are: the illegal drug trade, the port system, the city government and bureaucracy, the school system, and the print news media.

‘–30–’ has been traditionally used by print journalists to indicate the end of a story: it’s also the title of the final episode of The Wire.

There are many theories about how the usage originated. In my view, the strongest candidate theory is that back in the day when newspaper stories were written in longhand and transmitted to the newsroom via telegraphy, ‘X’ marked the end of a sentence, ‘XX’ the end of a paragraph, and ‘XXX’ meant the end of a story. The Roman numerals XXX translate to 30.

So, today my em-dashes are bifurcated.

They’ll be back again next time.

—30—

August 05 2010 | e-learning | Add a Comment »

Capturing non-formal learning-related data

Over the last month or so, I have been investigating approaches to evaluating non-formal learning in organizations.

This is the methodology I use to collect data on non-formal learning programs:

• Phase I: Distribute learning readiness surveys among stakeholders to capture a high-level organizational view of awareness and penetration within the organization
• Phase II: Capture quantitative data on as many discrete non-formal learning interventions as you can, preferably over a period of 6-12 weeks.
• Phase III: Distribute questionnaires to gather data from as wide a sample as you can from the population of organization employees who participated in the non-formal learning program.
• Phase IV: Conduct long-form (60-minute) follow-up face-to-face semi-structured interviews with all members of the Phase III cohort.

This four-phase approach uses both the qualitative and quantitative methods of data gathering I have outlined in my current series of blog posts, which investigate the subject of evaluating non-formal learning.

More next time…

–

August 04 2010 | e-learning | Add a Comment »

Is your Organization Ready for E-Learning?

Last week I discussed the value of e-learning readiness surveys as part of an overall strategy to evaluate the effectiveness of non-formal learning initiatives in organizations. The feedback on the survey was so good, I though that I’d delve a little deeper into why you there is training (and business) value in undertaking such a survey, even if only to get a better measurement of your organization’s e-learning capability.

According to Marc Rosenberg:

The questions provided in this survey represent some of the most important strategic issues organizations face when transitioning to e-learning. Certainly there are additional questions and issues that deserve attention; … add your own, organization-specific items.

(2000)

Now read on…

The questions are grouped into seven categories:

1. business readiness
2. the changing nature of learning and e-learning
3. value of instructional and information design
4. change management
5. (reinventing) the training organization
6. the e-learning industry
7. personal commitment to developing e-learning

Each category in the E-Learning Readiness Survey is as important as any other. According to Rosenberg “…a total of your responses is less meaningful than your own analysis of each individual item.”

Here are some things to think about when you review your responses, which are measured on a 0-5 Likert scale:

• Any item with a response of 0-1 could be a show-stopper. The low rating may indicate resistance to change, and/or a lack of readiness/capability on the part of people, infrastructure or the organization as a whole. Here is where you should concentrate your change management efforts to move the rating further to the right; your e-learning strategy won’t be successful until you do.
• Any item with a response of 2 or 3 indicates that progress is being made, but more effort would be worthwhile to avoid any disruption of your momentum or other unforeseen potholes. These are areas where opportunities for improvement may be easiest.
• Any item with a response of 4 or 5 indicates considerable progress. These are your success stories. Use them to help bring up the other areas of your strategy. For example, if you have strong senior management support, but you’ve sensed too much focus on technology alone, you might want to develop some communications from supportive managers that will help people understand the broader issues beyond the technology.

More…

–

August 03 2010 | e-learning | 1 Comment »