That got me thinking about why I’ve put tests in some of my courses and what functions they serve. Below are some questions that ran through my head. I’ve also put down my thoughts but I know that there are many other ideas out there and I look forward to hearing them.
Testing questions in my head:
1. Why do management and/or my client always want a test at the end of the course?
I don’t know about you but it seems that every time I build a course someone wants to put a test at the end. (Ok…sometimes it’s even me.) We always want to know if the learner understood and retained the information from the material. A text-based test at the end seems to be a quick and easy way to gain some level of confidence that knowledge transferred. Are there better ways to track this transfer and application back on the job? Sure, but it is more complicated than a simple end-of-course test. (But I’m sure we can all agree that the data gathered around how well learners are changing their behavior would be much more meaningful.)
2. Do we always have to test? Does every course need a test?
There are some courses that are fairly short and don’t tie to an essential part of someone’s job. Can a course like this not have a test? My thought would be yes. I guess it would boil down to why the course was created and do you need to “prove” to anyone that the time/resources were worth it. Maybe courses are joined together into Learning Paths and there is one final test for the entire path instead.
There are also times where we must meet compliance/regulatory requirements and we must be able to show test results. Paper-based test results can easily be presented in spreadsheets and charts.
Another great point that Gary mentioned was that “learners need to know they’ve passed”. Most of the time when we start a task we like to know when it’s completed. This thought could hold true with taking a course as well. We’d like some feedback/closure that we did what we needed to do. That our time was not wasted.
3. Is a simple True/False or Multiple Choice test ok?
Sure, if that’s the level of confidence you want to have that participants accomplished the course objectives.
4. Pre-test vs. Post-test…which one should I use? (maybe both?)
There are quite a few ways to create tests. Which option works best?
- Post-Test only
- Identical Pre and Post Tests
- Pre and Post Tests share the sames questions but randomized
- Different questions in both
5. Is the test score really an accurate indicator?
Some people are really good at remembering information and recalling that to find the correct test answers. Do they really change their behavior back on the job? Maybe the test score is more a reflection of how well the material was presented instead of actual learning.
6. Is there a way to test without “testing”?
If we put a bit more thought into what truly means success for our courses then we can discover other ways of measuring results than by just paper-based tests. For example, if we look at a sales course there are multiple things we could measure about each participant:
- Increase in sales 1, 3, 6 months following a course
- Surveys to the manager about sales concepts being applied
- Mystery shops / Observation of the participant in action
- Number of product referrals
- Increase in commission
- Customer satisfaction scores
Overall, I think that many times tests are thrown together quickly and don’t really gather meaningful data. Too often the success of our courses and participants is determined by test scores and often is not the best data to be using for that decision. With a bit more effort and brainstorming we should be able to move from Level 1 testing and get to 3 and 4.
I know that there are many aspects of testing that I’m missing and there is tons of information that I missed above. What are your thoughts? What questions run through your head?