Statistical tests catalog

All tests

Means (parametric)

Parametric tests assume a model for the data, usually the normal distribution, and compare its parameters, such as the mean. When the assumption holds, they are the most powerful.

• One-sample t · One sample

  Compare the mean of a sample with a reference value when the population standard deviation is unknown.

  H₀: The population mean equals the reference value: μ = μ₀.

• Two-sample t · Two independent samples

  Compare the means of two different groups, such as treatment and control.

  H₀: The two population means are equal: μ₁ = μ₂.

• Paired t · Two paired samples

  Compare two measurements taken on the same individuals, such as before and after an intervention.

  H₀: The mean of the differences is zero: μ_d = 0.

• One-way ANOVA · Three or more independent groups

  Compare the means of three or more independent groups at once, without inflating the type I error with several t-tests.

  H₀: All means are equal: μ₁ = μ₂ = … = μₖ.

• Repeated measures ANOVA · Three or more repeated measures

  Compare three or more conditions measured on the same individuals, such as several time points in a follow-up.

  H₀: The means of all conditions are equal.

• Tukey HSD · Multiple comparisons

  After a significant ANOVA, find out which pairs of groups differ, controlling the error of the whole set of comparisons.

  H₀: For each pair: μᵢ = μⱼ.

Ranks (nonparametric)

Nonparametric tests assume no distribution for the data. They replace each value by its rank, its position in order (1st, 2nd, 3rd…), which protects them from extreme values and makes them fit ordinal data.

• Sign test · One sample

  Test the median of a sample without assuming anything about the shape of the distribution; it only uses whether each value is above or below.

  H₀: The population median equals m₀.

• Mann-Whitney U · Two independent samples

  Compare two independent groups when the data are not normal, have extreme values or are ordinal.

  H₀: A value drawn from one group is equally likely to be larger or smaller than one from the other.

• Wilcoxon signed-rank · Two paired samples

  Compare two paired measurements without assuming the differences are normal.

  H₀: The distribution of the differences is symmetric around zero.

• Kruskal-Wallis · Three or more independent groups

  Compare three or more independent groups without assuming normality: it is the ANOVA done on ranks.

  H₀: All groups come from the same distribution.

• Friedman · Three or more repeated measures

  Compare three or more conditions measured on the same individuals without assuming normality.

  H₀: The conditions have the same distribution.

• Dunn’s test · Multiple comparisons

  After a significant Kruskal-Wallis, find out which pairs of groups differ.

  H₀: For each pair, the mean ranks are equal.

Proportions (counts)

For categorical answers, such as yes or no, hit or miss: the data are how often each category appears, and the test compares proportions.

• Binomial / one-proportion Z · One sample

  Compare the proportion in a sample with a reference value, such as a pass rate against 50%.

  H₀: The population proportion is p₀.

• Two-proportion Z · Two independent samples

  Compare the success rate of two independent groups, such as the conversion of two versions of a web page.

  H₀: The two proportions are equal: p₁ = p₂.

• McNemar · Two paired samples

  Compare a proportion measured twice on the same individuals, such as opinion before and after a campaign.

  H₀: The proportion did not change: switches in one direction and the other are equally likely.

• Chi-square of homogeneity · Three or more independent groups

  Compare the distribution of a categorical variable across three or more independent groups.

  H₀: The proportion of each category is the same in every group.

• Cochran’s Q · Three or more repeated measures

  Compare a proportion measured under three or more conditions on the same individuals, such as passes in several exams.

  H₀: The success rate is the same under all conditions.

• Marascuilo · Multiple comparisons

  After a significant chi-square test of homogeneity across three or more groups, find out which pairs of proportions differ.

  H₀: For each pair: pᵢ = pⱼ.

Variances (spread)

They compare how spread out the data are, not the center. Useful on their own, as in quality control, and to check the equal-variance assumption of other tests.

• Chi-square for one variance · One sample

  Test whether the variance of a process is at the specified value, as in quality control.

  H₀: The population variance is σ₀².

• F for two variances · Two independent samples

  Compare the spread of two independent groups.

  H₀: The variances are equal: σ₁² = σ₂².

• Pitman-Morgan · Two paired samples

  Compare the variances of two paired measurements, such as the precision of two instruments applied to the same samples.

  H₀: The two variances are equal: σ₁² = σ₂².

• Levene · Three or more independent groups

  Check whether two or more groups have the same variance, for example before an ANOVA.

  H₀: All variances are equal.

• Mauchly’s sphericity · Three or more repeated measures

  Check sphericity before a repeated measures ANOVA: the variances of the differences between conditions must be equal.

  H₀: The covariance matrix is spherical.

Goodness of fit and normality

Goodness of fit: do the data follow the expected distribution? Normality tests are the most common case and check an assumption of parametric tests.

• Shapiro-Wilk

  Check whether a sample comes from a normal distribution; it is the most powerful normality test for small and medium samples.

  H₀: The data come from a normal distribution.

• Kolmogorov-Smirnov

  Compare a sample with a fully specified theoretical distribution.

  H₀: The data follow the distribution F₀.

• Anderson-Darling

  Test goodness of fit to a distribution, giving more weight to the tails than Kolmogorov-Smirnov.

  H₀: The data follow the distribution F.

• Chi-square goodness of fit

  Check whether the observed frequencies in categories match the ones a theory expects, such as a fair die.

  H₀: The category proportions are the specified ones.

• Jarque-Bera

  Test normality through skewness and kurtosis; common in econometrics, with large samples.

  H₀: Zero skewness and kurtosis of 3, as in the normal.

• Lilliefors

  Test normality when the mean and standard deviation are estimated from the sample itself.

  H₀: The data come from some normal distribution.

• Grubbs

  Check whether the most extreme value in a sample is an outlier, that is, too far away to come from the same normal distribution as the others.

  H₀: There is no outlier: all values come from the same normal distribution.

Association and correlation

Instead of comparing groups, they measure whether two variables move together, and how strongly.

• Pearson correlation

  Measure and test the linear relationship between two quantitative variables.

  H₀: There is no linear correlation: ρ = 0.

• Spearman correlation

  Measure the monotonic relationship between two variables, even if non-linear or ordinal: it is Pearson computed on ranks.

  H₀: There is no monotonic association.

• Kendall’s tau

  Measure the agreement in order between two variables, counting concordant and discordant pairs.

  H₀: There is no association: τ = 0.

• Chi-square of independence

  Check whether two categorical variables are associated, in a contingency table.

  H₀: The two variables are independent.

• Fisher’s exact

  Test association in a 2 × 2 table with small counts, where the chi-square test does not hold.

  H₀: The two variables are independent.

• Point-biserial correlation

  Measure the relationship between a binary and a quantitative variable; it is equivalent to the two-sample t-test.

  H₀: There is no correlation.

• Cohen’s kappa

  Measure the agreement between two raters who classify the same items into categories, discounting the agreement expected by chance.

  H₀: The agreement is only that of chance: κ = 0.

Models and regression

Tests run inside a fitted model: whether a coefficient matters, whether the model explains anything and whether the residuals meet the assumptions.

• t for a coefficient

  Test whether an explanatory variable has an effect in a regression, holding the others fixed.

  H₀: The coefficient is zero: βⱼ = 0.

• Overall regression F

  Test whether the regression model as a whole explains anything, or whether all coefficients may be zero.

  H₀: All coefficients (except the intercept) are zero.

• Two-way ANOVA

  Assess at once the effect of two factors on a numeric response, and whether they interact, such as dose and sex in a trial.

  H₀: For each factor, the level means are equal; for the interaction, the effect of one factor does not depend on the level of the other.

• Likelihood ratio

  Compare two nested models fitted by maximum likelihood, as in a logistic regression.

  H₀: The smaller model is enough.

• Wald

  Test a coefficient of a model fitted by maximum likelihood using only the full model.

  H₀: The coefficient is zero.

• Hosmer-Lemeshow

  Check whether the probabilities predicted by a logistic regression match the observed frequencies, grouping the cases by risk band.

  H₀: The model fits well: observed and predicted frequencies agree.

• Durbin-Watson

  Check for first-order autocorrelation in the residuals of a regression, common with data over time.

  H₀: The residuals have no autocorrelation.

• Breusch-Pagan

  Check whether the variance of the residuals of a regression is constant (homoscedasticity).

  H₀: The residuals are homoscedastic.

Time series

For observations ordered in time, where neighboring values tend to be correlated: they test stationarity and autocorrelation.

• Augmented Dickey-Fuller

  Check whether a time series has a unit root, that is, whether it is non-stationary.

  H₀: The series has a unit root (it is not stationary).

• KPSS

  Check the stationarity of a series with the null hypothesis reversed relative to the ADF; the two together complement each other.

  H₀: The series is stationary.

• Ljung-Box

  Check whether a series, or the residuals of a model, still have autocorrelation over several lags.

  H₀: The autocorrelations up to lag h are all zero (white noise).