Now that you have prepared quality content, you need to ensure that students are really learning. This can be done through the use of assessments and evaluations. A combination of both tools will help to make teaching online more manageable and self-directed for the student.
Learning Outcomes
At the end of this module faculty will be able to:
Differentiate between assessment and evaluation.
Define formative and summative assessments and evaluations.
Understand the elements that are required for an assessment or evaluation tool to be effective.
Identify assessment and evaluation tools.
Build effective test questions.
Define rubric and identify the differences between a holistic and analytic rubric.
Build a rubric for an assessment or evaluation.
Implement the elements of quality feedback.
Use the Review features of Microsoft Word to provide students with feedback.
When reading the educational literature in this area, the terms assessment and evaluation are often used interchangeably. This can be confusing to students.
The table below highlights some of the key differences.
Assessment
Evaluation
Emphasis on the teaching process and progress
Emphasis on the mastery of competencies
Focus on the Teacher Activity or Student Activity
Focus on Student Performance or Teacher Performance
Methods include: Test/Quizzes, Semester Projects, Demonstrations or Performances
Purpose is to improve the teaching and learning process
Purpose is to assign a grade or ranking
Generally Formative
Generally Summative
Source: Adapted from LENS (League for Innovation): Assessment and Evaluation Modules
Assessments and evaluations are valuable elements in the learning process. They provide evidence that learning is happening through a progressive process and ultimately in the allocation of grades. Assessments and evaluations are excellent vehicles for providing feedback and for helping students to be self-directed in the learning process.
While some form of feedback should be provided in all assessments and evaluations, grades are most often associated with evaluations. Your online course(s) should provide a combination of self-assessments (where the student completes an activity and receives immediate feedback) and instructor-assessed activities (where the instructor provides direct feedback to the student - can be an assessment or an evaluation). Instructor-assessed activities require you to provide more than a grade; you should also give the student some direction that will help them improve on future activities. More information about this can be found in this module’s Feedback section.
When building assessments and evaluations in your course, it is important that you:
Adequately measure the learning objective.
Include clear instructions on how to complete the tool/activity and give specific information about how the activity will be assessed or how it contributes to the final grade for the course.
Include questions or scenarios that are relevant to the learning material at that particular stage of the course.
The OntarioLearnCourse Development Checklist provides specific details of how assignments should be presented in your online course. Be sure to click the "Learn More" links for additional detail.
a) Assignments are sequenced logically within the Assignment section Learn more...
b) Assignments are sequenced in the order used in the Course Schedule/Timeline. Learn more...
c) Due dates are specified for each assignment (can be in the Course Schedule/Timeline) Learn more...
d) Instructions on how to submit assignments are provided. Learn more...
e) Learners are provided with the criteria upon which the assignment will be evaluated. Learn more...
f) All assignments are free of mechanical errors. Learn more...
Things to Consider When Creating Assessments And Evaluations
In most courses, the evaluations for your course will be predetermined by the specifications in the course outline. You have a little more flexibility in relation to the assessments you add to your course site. Regardless of building an evaluation or assessment tool, you should try to include a variety of approaches. Students can become quite bored with the "same old activities".
Good Resource:
The following link will take you to the Penn State - learning community (wiki) that shares a variety of resources to help you build meaningful test questions that will contribute to the creation of an effective assessment or evaluation.
Multiple Choice tests, while they take more time to develop, are one of the most common testing methods used in the online environment. The advantages of multiple choice is clearly in the objectivity of the grading and the familiarity students have with this method. Research shows that multiple choice tests are effective for testing comprehension, application, content analysis and objective problem solving; however, synthesis, evaluation and demonstration of writing skills cannot be tested this way.
When making your decision on what types of assessments to use in your course, keep the following things in mind:
The method should have a relationship to specific learning outcomes: You must ensure the assessment or evaluation contributes to the goal of meeting the stated course or lesson outcomes.
The value of the evaluation in the overall grade: The amount of work you should expect from the student should correlate to the value the evaluation has in the overall grade for the course.
Location of the assessment or evaluation in the sequence of the course: Building on a previous assessment or evaluation (or scaffolding) is a good strategy. It helps the students retain the content learned and reuse it in subsequent assessments or evaluations.
Providing constructive feedback to students is a critical element of good teaching practice. Specifically in the online environment, feedback helps to minimize the isolation factor and encourages student participation in the course. Research has shown that feedback is more strongly related to student success than any other teaching behaviour. It is clearly a skill that all teachers should have and develop.
Good Resource:
Providing Students with Effective Feedback authored by Kathy Dale goes more in-depth into this concept and provides strategies that you can incorporate into your teaching.
The Power of Timely Feedback
Providing feedback in a timely manner tends to reduce the anxiety level students have around completing assessments and evaluations. It also shows students that you are actively engaged in the course and are interested in their success.
If you include formative strategies (activities/assignments that build upon each other) in your course, you need to ensure you provide feedback within a timeframe that allows students to incorporate your comments/suggestions into their work.
When you create your assessments or evaluations, you should clearly state in the instructions when a student can expect to receive feedback. Depending upon the size of your class and your teaching practice, it is a good idea to state that assignments will be returned 1 week after the posted due date. If there are instances when this may vary, you need to communicate this to your students.
Elements of Feedback - more than a score!
Never underestimate the powerful impact of personalized feedback. It shows the student that you took the time to review their work and provide specific feedback. This is a motivational factor for your students and helps to build upon the student-instructor relationship.
As noted in the Dale article linked above, researchers Black and William outline three key elements in "enhanced feedback":
Recognition of the desired goal.
Evidence about level of achievement in relation to the goal.
Information on how the student can improve and meet the desired goal.
Consider the following example: Students are asked to prepare a memo advising the teacher of their chosen topic for a final research project. In this memo they are asked to:
Use the appropriate format for a business memo.
Clearly state the title they have chosen for the report.
Indicate how the report will enhance their understanding of a specific topic addressed in the course.
Cite three potential resources they will use in their research.
When a teacher reviews the submission, all of these elements should be presented in order to meet the "desired goal". If certain elements are missing in the submission, the teacher's feedback should:
Reinforce the requirements.
Clearly identify what information is missing.
Provide information on how to improve.
Here is a sample of a response to a student:
"John, while you demonstrated a good understanding of the format of a business memo, you did not include the required information. It is not clear to me how your choice of topic relates to one of the subjects we discussed in this class. While your two sources are relevant to your topic, you will need to find additional sources to support your position going forward. I recommend that you review the resources available at www.anexample.ca and visit the college's library to find other suitable sources."
Rubrics are another excellent way to provide your students with feedback.
Rubrics are simply defined as a rating scale used to evaluate a student's ability to complete specific objectives that are measured using competency or performance criterion.
Using rubrics is beneficial to both you and the student. When built properly, they clearly articulate to the student what is expected in a particular assignment and defines what a student needs to do to demonstrate a competency. A rubric also gives the teacher a very objective method for providing feedback while creating a frame of reference for specific comments about a student's work.
Time Management Tip:
Adding a rubric helps clarify assignment instructions and submission requirements. It can also limit the number of questions like "what do you want me to include in this assignment?" that you may get too!
There are generally two types of Rubrics
Holistic
Analytic
Holistic Rubrics
Provide students with a single score based on the overall performance of a particular task or assignment. Click the Example: Holistic Rubrics below to see an example.
Example: Holistic Rubrics
Score
Description
5
Demonstrates complete understanding of the problem. All requirements of task are included in response.
4
Demonstrates considerable understanding of the problem. All requirements of task are included.
3
Demonstrates partial understanding of the problem. Most requirements of task are included.
2
Demonstrates little understanding of the problem. Many requirements of task are missing.
Break the evaluation into a series of measurable objectives, which are scored upon a continuum of performance indicators. Click the Example: Analytic Rubrics below to see an example of a rubric to evaluate student participation in class discussions (created as a grade form in Blackboard).
Example: Analytic Rubrics
Criteria Performance Indicators
Objective
Unsatisfactory/
Incomplete
Needs
Improvement
Meets
Expectations
Exceeds
Expectations
Grammar and Spelling
(0 points)
(1 points)
Frequent misspellings, basic grammar/spelling errors that impair clarity and readability.
(2 points)
The few grammar and spelling mistakes do not impair clarity.
(3 points)
Entry is grammatically perfect and spelling-error free.
Substantive Content
(0 points)
(1.5 points)
Minimal content with weak response to prompt.
(2 points)
Adequate and appropriate content, with attention paid to prompt.
(3 points)
Superior content that exceeds response to prompt and/or stimulates discussion.
Readability
(0 points)
(1 points)
Paragraphing would be appropriate but is missing.
(1.5 points)
Paragraphing sufficient.
(2 points)
Highly readable, with sufficient white space, clearly organized paragraphs and graphic highlighting if possible.
Response to Others
(0 points)
Does not comment or respond to any other post.
(1 points)
Responds to one student but response is one-sentence or minimal.
(1.5 points)
Responds to one student courteously and thoughtfully.
(2 points)
Responds to more than one student courteously, extensively and thoughtfully.
TOTAL
/10
Source: Humber Business Communications - Discussion participation Rubric (created as a grade form in Blackboard)
Things to consider when building a rubric.
Creating an effective rubric can be very time consuming at the start; however, adding these to your course site will limit the number of questions you will get about "What is exactly expected in this assignment?"
Penn State has prepared a checklist of things an instructor should consider prior to building a rubric. Review this checklist.
Ways to create a Rubric
There are 3 ways to create a rubric; each has a varying degree of technical difficulty.
Build your own (using a table in Word) and include it in your assignment instructions (low)
The Review Tool bar provides quick access comment feature, as well as the Track Changes option.
There are a variety of tools within Microsoft Office that can be used when reviewing student work. You can insert text in a different colour or use text boxes or shapes to provide comments; however the review features in Word are by far the most efficient.
Depending on the version of Office you are using, you will find the Review tab on the main toolbar or you may have to add the Review toolbar through the VIEW tab (View>Toolbars>Review Toolbar).
The Review toolbar provides a quick access comment feature, as well as the Track Changes option.
The Review Tool bar provides quick access comment feature, as well as the Track Changes option.
Good Resource:
For more specific instructions review the Track Changes Demo on the Microsoft website.
What is Track Changes?
It is a feature that allows you to edit (make changes to) a document without impacting the integrity of the original one, until the original author accepts the change. This feature is available in MS Word and Excel.
Track Changes presents your edits in a different colour and includes a specific notation of the change in the right margin. If the student wishes to accept your changes, s/he can do so by clicking "Accept Changes" in the Review toolbar. Accepting your changes will remove all your markings and the changes are automatically applied to the document.
Adding Comments: This is the easiest and most recommended way to give feedback on assignments submitted using Microsoft Office applications. This feature is available in Word, Excel and PowerPoint.
To use this feature, you simply select (highlight) or click on the specific content in a document you wish to comment on. Then, click Add Comment in the Review Tab. This will produce a text bubble in the right column of the screen where you can type your comment.
Good Resource:
For more specific instructions review the Comments article on the Microsoft website.
Sometimes it may be more effective to provide verbal feedback. The Insert Voice option in Word allows you to do just this. To use this feature you will need access to a good microphone. Experience shows that a headset microphone works best in these instances.
For more detailed instructions on how to do insert audio comments visit MS Office.
TIP:If you intend to use this feature be sure to let your students know. While this shouldn't present an issue, itis important for students to know that they will need access to speakers to hear your comments.
If you have not already done so, add the Insert Voice feature on your Quick Access toolbar in Word. Follow these steps to do this:
Click the Office Button.
Click the Word Options button.
Click Customize.
Click the drop-down arrow of the Choose Commands From box and select Commands Not In Ribbon.
Scroll to and select Insert Voice.
Click the Add button, and then click OK.
The Insert Voice button will be available for you in the Quick Access menu at the top of the screen.
Time Management:
Create a general comments document or "Cheat Sheet". This can include a list of common feedback statements or messages that can be copied and pasted as comments.
Helpful Feedback Tips
If you are using the LMS drop-box feature to return assignments, create a general message that can go in the comment box that advises the student to review the returned file.
Please review my feedback in the attached GRADED file.
Good work on this assignment. You have clearly demonstrated that you know how to create a table in Word.
Don't forget the personal touch: While the general comments provide an excellent starting point for your feedback, you may need to make some minor personalized edits.
Save a copy of the assignment after you have reviewed it: Be sure to rename it so that it denotes the assignment has been GRADED or REVIEWED. E.g. (JSmith_Assmt1_GRADED)
Create a folder system for all assignment submissions. This makes it easier to locate the reviewed files when returning them or for future reference should a student make an inquiry about your feedback.
The terms "Assessment" and "Evaluation" have been used interchangeably in some literature, but the basic distinction is that "evaluation" is used to provide a grade or score, and "assessment" is used as a measure of progress. Both are important in the learning environment.
As a good online teacher, you need to be able to provide meaningful and timely feedback to your students. This does not mean that you have to be grading every activity that is present in your course site. Students should be able to complete some assessments and receive objective feedback immediately (e.g. using learning objects or self-assessed quizzes).
Building effective assessments and evaluations requires a solid understanding of the learning outcomes for your course. Assessments can help keep the student engaged and also give them indirect feedback on the comprehension of the material to that particular point.
As you are well aware, student participation in the online environment is typically driven by "what is graded" - the evaluations. Students want to make sure that they meet the end goal of passing the course. It is your challenge to build evaluations and assessments that are meaningful to the overall learning process and provide students with feedback to explain how they have been graded - a score or grade is not enough.
Next steps:
There has been so much growth in the variety of technology that is available to teachers today; it is hard to keep up. Similar to creating assessments and evaluations and providing meaningful feedback, a good online teacher must always keep the student at the forefront of any decision making process. Module 4 will explore various ways you can utilize the existing LMS tools and explore other web-based tools that you can use to build content or encourage interaction.
Another way to differentiate between assessments and evaluations is to understand the difference between the terms Summative and Formative.
Just as the spelling of the words suggest FORMative has the focus on how the knowledge or learning is FORMed throughout the course. SUMmative has the focus on the SUM of the performance of specific evaluations.
A good online course will use both Summative evaluation and Formative methods.
Summative
Formative
For grades
Enhance learning
Usually occurs at critical points in the learning process (e.g. mid-term; final exam)
Considered a part of the course instruction
Evaluated with a score
Evaluated by providing feedback
Once an evaluation is complete, it is added to the students record; typically no opportunity for change
Activities tend to build upon the learning process (i.e. tasks will flow into each other so learning becomes more of a process)
Can be viewed as "threatening" as the end result is more definitive
option in Word allows you to do just this. To use this feature you will need access to a good microphone. Experience shows that a headset microphone works best in these instances.