Purpose:

This page is a collection of the resources and basic methodology that I use when helping schools, teachers, and course designers evaluate their online courses and workshops. When developing a course, especially an online course, having an extra set of eyes can be a necessity. And having that extra set of eyes take a systematic and methodical look at the course can be a great advantage. I use this page as my personal “Course Evaluation Toolkit” as such it is a brief overview and collection of links that I use.

Methods and Types of Evaluations:

“Informal Review” – When I think of an informal review I think of a quick run through with a second set of eyes sometimes with a rubric, sometimes without.  Having someone who has completed formal reviews can help since they can usually spot areas of improvement quickly.

“Formal Review” – A formal review is a more detailed and systematic review of an online course by an expert in online education and typically uses a rubric and ends with a formal report with suggestions for improvement.

“Formative Evaluation” – A pilot study or focus group is often the most effective method for a formative evaluation of a course. Actual students, or a group who can approximate the student mindset, move through the course and actually complete the activities and are then either surveyed or interviewed for their feedback.

“Peer Review” – A peer review is usually done after the course has been through at least a pilot study, after the primary instructor/designer has gone through, and after a few students have gone through the course. The peer review should include about three reviewers who will systematically review the course using rubrics similar to the formal review. While the reviewers do not need to be experts in the content of the course it is helpful if at least one is.

“Summative Evaluation” – There are many ways to handle a summative evaluation. My preferred method is to evaluate the success and satisfaction of the students who have completed the course. Did the students learn what they were supposed to learn? Are the students able to apply that knowledge/skills? Ultimately the final determination of a course’s success is the success of the students who complete the course.

Rubrics Make the Evaluation Easier:

For the most part, I find the Quality Matters Rubrics to be the most useful and their professional development is not only a fantastic resource for developing and evaluating online courses and workshops they are also fantastic examples of well planned out online learning products.

The annotated Rubrics are worth the effort of the workshops, however, there are un-annotated copies of the quality matters rubrics available for free:

The rubrics are also useful tools to use during the design process since they can be used as a checklist of key best practices.

The annotated rubrics are available for members, currently, the fee for an individual membership is approximately $175 a year per rubric.

https://www.qualitymatters.org/qm-membership/benefits-fees

There are other rubrics available as well that I use often.

The University of Illinois has a useful rubric that they have released under a Creative Commons Licence which makes it free to use.

Their web-based rubric provides an excellent overview of best practices that can help with the design of online courses and they offer downloadable copies with room for comments and a checklist which can help when giving feedback on an already developed course.

http://www.ion.uillinois.edu/initiatives/qoci/rubric.asp

The University of Southern Mississippi Learning Enhancement Center has an “Online Course Development Guide and Rubric” It is essentially a rubric that I also like, while it doesn’t have as many specifics as the others it is an excellent way to quickly communicate a course evaluation.

http://ablendedmaricopa.pbworks.com/f/LEC_Online_course+rubric.pdf

Michigan Community College Association has another rubric. One of the items I like about this rubric is that it covers the importance of support for the faculty as well as the importance of “course maintenance” which is not common in other rubrics.

http://www.mccvlc.org/~staff/content.cfm?ID=108

Clayton Wright from Grant MacEwan College has a detailed outline that works as an excellent checklist for evaluating courses. It is quite comprehensive with over 100 points to check and can be used for a detailed review of courses.

http://elearning.typepad.com/thelearnedman/ID/evaluatingcourses.pdf