Course Evaluation

Evaluation is a measurement tool. Effective evaluation will provide invaluable information which you can use to accomplish the desired goals and objectives. Create a plan as to how you're going to evaluate the effectiveness of your instruction. Some items to consider when creating your plan are:

what you're going to evaluate
when you're going to evaluate
how you're going to solicit and receive responses
how you're going to analyze your data
During the design and creation of your course keep a close watch, to ensure that your materials and methods are in keeping with the goals and objectives that are set. Doing this will reduce the number of problems you will see during implementation. As implementation is the first real test to determine effectiveness, a "dry-run", or pre-test, is ideal. Factors beyond your control, such as deadlines, pressure from the powers-that-be (whether they be management or clients), or what have you, may prevent a pre-test. In this case, the first full run of the course is also your test.
During implementation evaluate such things as:

How clear is the material?
How is your effectiveness as an instructor?
Is student participation to your expectations?
How can you make the course better?
Do the assignments and exams go with the material?
An outstanding write-up of different types of evaluation can be found at University of Idaho. Here is what they say about two forms of evaluation, formative and summative:
Formative evaluation:

Is an ongoing process to be considered at all stages of instruction.
Will enable the instructor to improve the course as he/she proceeds.
Facilitates course and content adaptation.
Will identify major gaps in the instructional plan or the need for minor adjustments.
Summative evaluation:

Assesses overall effectiveness of the finished product or course.
Can be a springboard in developing a revision plan.
Can be a baseline of information for designing a new plan, program, or course.
Will not help current students since it is conducted upon course completion.
More information on this can be found in these documents, University of Idaho's Guide 3 and University of Idaho's Guide 4. These are a part of their Distance Education At A Glance collection.
Using the UNIX course as an example:

The course text, assignments, and exams were online
Calls could be made to my house after 1800 (6:00pm), but not to my place of work (I had another full-time job).
I was willing to visit the client's site to provide instruction.
Electronic mail would be used for the submission of assignments, to ask questions, and to answer questions.
The class was self-paced, but had to be completed in 15 weeks.
According to the requirements, the course was a success. All of the requirements were met. This doesn't mean that it wasn't without problems. For example:

Many students did not direct many questions to me. Rather, they asked others in the workplace who had UNIX experience.

some parts of the text wasn't clear.
Some of the students were uncomfortable with the online nature of the course.
It's easier to ask someone who is already on site (this can also be a benefit).
... and other reasons I may not be aware of.

The Internet provider changed physical locations in the middle of the course. As a result, mail was not steady for about a week and many of the assignments were lost.
There were equipment configuration problems during the first two weeks of class.
Some of my assignment and test questions were vague.
... and there were other problems.
Many problems were addressed as the course progressed. For example:

Working with one of the students we found a configuration problem on one of the classroom computers. At the same time we found that I had sent a text file to the Web server as a binary file. Both of these were corrected.
Vague exam questions and assignments were re-worded and a class-wide mail was sent. If a test question affected a grade, the grade change was made in favor of the student.
Answers to questions that were called in or mailed were given directly to the student. A Frequently Asked Questions (FAQ) page was created. The question and answer would be posted for the entire class. The name of the person asking the question would not appear in the FAQ. Sometimes, a class-wide mail was sent instead, again, without the name of the person asking the question.
You can see, basically, how evaluation was carried out as the course progressed. The course ended with good marks, but could I make it better?
You bet.

The FAQ and e-mail answers can be used in the course rewrite.
The course worked well as a self-paced course, but it could easily be used in a classroom. If I were not teaching the classroom protion of the class, a facilitator could answer questions immediately. Those questions and answers could be relayed to me and the FAQ updated.
Students not familiar with Web-based technology could get one-on-one help with an onsite facilitator.
A listserv was not used in the first run of the course. A listserv would have been a great way to stir interaction with the students.
Notice that there were equipment problems as well. Evaluate the logistics and support required to make the course available.
These points do not cover everything; there is much more that was found and corrected as well as how the course could be improved. This should give you a few ideas in how you may want to create an evaluation plan. Look for the things that are working right as well as problems. Work with the information always making improvements to your course material and instruction.

Move directly to:

Site Home Page
Web-based Training Introduction Page
Table of Contents

Last Modified: February, 2005