Share Paper: Assessment Issues in an On-Line Environment - Addressing Rater Reliability Issues

  1. Jackie McBride, Arkansas State University, United States
  2. Joan Henley, Arkansas State University, United States
  3. Mary Jane Bradley, Arkansas State University, United States
  4. Joe Nichols, Arkansas State University, United States
  5. Annette Hux, Arkansas State University, United States
  6. Cindy Nichols, Arkansas State University, United States
Thursday, March 29 4:15 PM-5:15 PM Edison Ballroom D

Abstract: Assessment in a large scale on-line environment has potential reliability issues when two or more independent assessors are scoring assignments Determining the consistency of the scoring across the assignment can be challenging Ensuring that students are not subject to the luck of the draw for the assessor requires preparation of the assessors before the scoring begins The assessment process is subject to error from an assessor who is not prepared or who incorrectly interprets scoring guidelines Interpreting the performance of students with confidence and reliability among assessors is possible when all assessors are prepared for scoring in a consistent manner ...