What is inter-rater reliability?

Inter-rater reliability is a measurement of how much two judges or raters agree in their assessments. Two humans do not generally rate or assess a single item in the same way, which makes inter-rater reliability useful.

Reliability tests assess whether results of any assessment are both stable and consistent. Inter-rater reliability is only useful when something or someone is assessed by two or more people.

Inter-rater reliability is often used when artifacts, portfolios or other subjective items are being judged, as it compares the opinions of the raters. For strictly objective measurements, retests and alternative forms of the measurement are used.