Fisher’s exact test
Association and correlation · reference distribution: Hypergeometric
When to use it
Test association in a 2 × 2 table with small counts, where the chi-square test does not hold.
Null hypothesis
The two variables are independent.
Assumptions
- Fixed margin totals (or treated as fixed): the p-value adds up the tables with those margins that are as likely as the observed one or less
Test statistic
P(T) = \dfrac{\binom{a+b}{a}\binom{c+d}{c}}{\binom{n}{a+c}}, \qquad p = \sum_{P(T') \,\le\, P(T_{\text{obs}})} P(T')Effect size
Odds ratio of the 2 × 2 table.
OR = \dfrac{a\,d}{b\,c}How to report it
OR = 6.4, p = .028 (Fisher’s exact test, two-sided)
In R and Python
R
fisher.test(table)Python
stats.fisher_exact(table)In Python, stats is scipy.stats and np is numpy.
Variants and alternatives
- Chi-square of independence, with larger counts
Where it sits in the catalog
Association and correlation. Instead of comparing groups, they measure whether two variables move together, and how strongly.
In the decision tree
- What do you want to do? Compare groups, or one group with a reference value
- What kind of response did you measure? Categorical: yes or no, or categories
- How many groups or measurements? Two independent groups
- Are all expected counts at least 5? No
- What do you want to do? Measure the relationship between two variables
- What kind of variables are they? Both categorical
- Are all expected counts at least 5? No, in a 2 × 2 table