Scoring validity: Designing and implementing an online training module for rater training

ID: 50039 Type: Roundtable
  1. Cristina Perez-Guillot and Julia Zabala-Delgado, Universitàt Politecnica de València, Spain

Tuesday, March 7 4:15 PM-5:15 PM

No presider for this session.

The reliability of direct testing of writing has been a concern since the 19th century (Edgeworth 1890). Although variability in rating writing is an inextricable part of the process inherent to its subjectivity, random variability can lead to casualization of marker consistency (Barrett, 1999) and have devastating consequences for students. Training teachers to score scripts is presented in the literature error as the solution to reduce rater error while respecting the individuality of both the rater, the student and the task at hand. However, although some authors recommend an entire rater training day and cyclic training sessions. The training duration is generally limited. Online training modules are presented as the solution to lack of time and resources on the part of teachers and institutions. Consequently, an online training module was designed based on the guidelines given by Weigle (2002) to improve both inter and intra rater reliability as well as to reduce rater errors


Our workshop intends to highlight the importance of training teachers for assessing writing to achieve both reliability and validity in their judgements. The limitation of time and resources faced by teachers and institutions calls for alternative forms of instruction and the workshop gives the opportunity of experiencing an online methodology designed with this purpose in mind.

Topical Outline

Experiencing rater-training modules for teachers of English to help their reliability and validity when assessing student’s writings


Teachers of English with some experience in assessing writing

Experience Level



Dr. Cristina Pérez-Guillot, she is currently Senior Lecturer of Business English at Universitat Politècnica de València, Department of Applied Linguistic, Head of Language Centre of The Universitat Politècnica de València (Spain) Research areas: Assessment and Testing of English as a Second Language, Development of evaluation tools, e-learning / teaching English as a second Language, / Development of Teaching Material Dr. Julia Zabala Delgado, MA in Language Testing at Lancaster University. Language Advisor at the Language Centre of the Universitat Politècnica de València coordinating standardized exams and test development and rater training. Her main areas of interest are Language Testing, Development of Assessment Tools, Translation studies


Conference attendees are able to comment on papers, view the full text and slides, and attend live presentations. If you are an attendee, please login to get full access.