⚠️ This is a simplified guide to help you quickly pick a suitable tool, not a full course on statistical methods. For more complex situations (multi-factor designs, non-parametric tests), consult your course or instructor.
- Quickly finding direction when unsure which statistical test to use
- Reviewing when different statistical methods apply
- Teaching or self-studying introductory statistics
The three questions that decide the test
Choosing a statistical test looks like it requires memorising a large table. It mostly reduces to three questions, asked in order:
- What kind of question am I asking? Is there a difference between groups? Is there a relationship between variables? Do observed counts match what I expected?
- What kind of outcome do I have? Continuous (heights, scores, times) or categorical (pass/fail, party voted for)?
- How many groups, and are they independent? One group against a known value, two groups, or three or more? And are the same subjects measured twice, or are they different people?
Those three answers almost always leave exactly one standard test standing.
The decision table
| Your question | Outcome | Design | Test |
|---|---|---|---|
| Differs from a known value? | Continuous | One group | One-sample t-test |
| Do two groups differ? | Continuous | Independent | Two-sample t-test |
| Did they change? | Continuous | Same subjects twice | Paired t-test |
| Do 3+ groups differ? | Continuous | Independent | One-way ANOVA |
| Are two variables related? | Both continuous | Paired observations | Pearson correlation |
| Are two variables related? | Both categorical | Cross-tabulated | χ² independence |
| Do counts match expectations? | Categorical | One variable | χ² goodness of fit |
Everything in the right-hand column is available on the hypothesis test page, and the wizard above links straight to it with your test preselected.
Paired or independent — the distinction people get wrong
This is the most consequential fork in the table, and the easiest to misread. The question is not whether the two sets of numbers are related in some general sense; it is whether each observation in one set has a specific, meaningful partner in the other.
- Paired: the same students before and after a course. The same patients on two drugs. Twins, one in each condition. Every row is one entity measured twice.
- Independent: class A against class B. Treatment group against control group. Different people in each, and no row-by-row correspondence.
If your two columns must have the same number of rows for the data to make sense, the design is paired. Getting this wrong is costly in both directions: analysing paired data as independent throws away the precision the pairing bought you and usually loses a real effect, while analysing independent data as paired is simply invalid.
When the standard test does not fit
The tests in the table are parametric — they assume roughly normal distributions and, for the group comparisons, similar variances. When those assumptions clearly fail, the usual replacements are:
| Parametric test | Non-parametric alternative |
|---|---|
| Two-sample t-test | Mann–Whitney U |
| Paired t-test | Wilcoxon signed-rank |
| One-way ANOVA | Kruskal–Wallis |
| Pearson correlation | Spearman's rho |
This site does not currently implement the non-parametric tests. If your data are strongly skewed, ordinal, or from a very small sample, use R, jamovi or SPSS for these rather than forcing a parametric test here. Saying so is more useful than offering a test that would quietly mislead you.
Decide before you look
Pick your test from the design of your study, not from the results. Running several tests and reporting whichever crossed p < 0.05 invalidates the p-value — the whole logic of the threshold depends on the test having been fixed in advance. If you genuinely cannot decide between two defensible analyses, report both.
The same applies to the one-tailed / two-tailed choice, to whether outliers get excluded, and to which covariates go into a model. Every one of those is a researcher degree of freedom, and deciding them after seeing the data is what makes so many published findings fail to replicate.
After the test
A p-value is half the answer. Once you have run the test, convert the result into an effect size so you can say how big the difference is, not just whether it cleared a threshold. And if the result was non-significant, check with the power analysis tool what effect your sample could realistically have detected before concluding there is nothing there.
Sources
- Field, A. (2017). Discovering Statistics Using IBM SPSS Statistics (5th ed.). Sage.
- Simmons, J. P., Nelson, L. D. & Simonsohn, U. (2011). False-positive psychology. Psychological Science, 22(11), 1359–1366. (On researcher degrees of freedom.)
- Wasserstein, R. L. & Lazar, N. A. (2016). The ASA statement on p-values. The American Statistician, 70(2), 129–133.