Cohen's Kappa

= −

Tap a term to see what it means, or .

Calculate

Example: two radiologists rated the same 100 scans as normal or abnormal. They agreed on 40 normal and 45 abnormal; on 15, they disagreed. How much do they agree beyond what chance would explain? Change the values to work out your own case.

Agreement table Rows: rater 1's categories; columns: rater 2's, in the same order. Rows separated by semicolons.

Step by step

What each term means

Kappa (κ): Agreement beyond chance, from −1 to 1: 1 is perfect agreement, 0 is what two raters guessing with the same frequencies would get, and negative values are worse than chance.
Observed Agreement (pₒ): The share of cases where both gave the same rating: the sum of the diagonal divided by the total.
Chance Agreement (pₑ): The agreement that would come from each rater's frequencies alone, if they rated independently. If both call almost everything normal, they agree a lot even without any criterion.
Room Beyond Chance (1 − pₑ): How much agreement there was left to win beyond chance. Kappa is the share of it that was achieved.

When to use it, assumptions and alternatives: Cohen's kappa in the statistical tests catalog.