Cochran’s Q test
Proportions (counts) · Three or more repeated measures · reference distribution: χ²(k−1)
When to use it
Compare a proportion measured under three or more conditions on the same individuals, such as passes in several exams.
Null hypothesis
The success rate is the same under all conditions.
Assumptions
- Binary response
- Independent blocks (individuals)
Test statistic
Q = \dfrac{(k - 1)\left[k \sum_j C_j^2 - N^2\right]}{kN - \sum_i R_i^2}How to report it
Q(2) = 7.6, p = .022
In R and Python
R
library(rstatix)
cochran_qtest(df, response ~ condition | id)Python
from statsmodels.stats.contingency_tables import cochrans_q
cochrans_q(matrix) # rows = individuals, columns = conditionsIn Python, stats is scipy.stats and np is numpy.
Variants and alternatives
- McNemar, the case with two conditions
Where it sits in the catalog
Proportions (counts). For categorical answers, such as yes or no, hit or miss: the data are how often each category appears, and the test compares proportions.
Three or more repeated measures. The same individuals measured under three or more conditions or time points: the paired design with more than two measurements.
In the decision tree
- What do you want to do? Compare groups, or one group with a reference value
- What kind of response did you measure? Categorical: yes or no, or categories
- How many groups or measurements? Three or more measurements on the same individuals