⚠️ These are general-purpose statistics tools, not tied to any specific exam's admission data. Good for homework, projects, or getting a feel for statistical testing basics — for formal academic work, follow the method your course/instructor specifies.
Significance at common α levels
| α level | Result |
|---|
Descriptive statistics
| Statistic | Value |
|---|
- Getting a quick p-value for homework or a project
- Checking a dataset's central tendency and spread
- Cross-checking output from other statistics software
What a p-value is, precisely
The p-value is the probability of obtaining a result at least as extreme as the one you observed, assuming the null hypothesis is true. That conditional clause is the whole thing, and dropping it produces almost every common misinterpretation.
In particular, a p-value is not the probability that the null hypothesis is true, and 1 − p is not the probability that your hypothesis is correct. It is a statement about how unusual your data would be in a world where nothing is going on — not about how likely that world is. The American Statistical Association issued a formal statement on this in 2016 precisely because the misreading is so widespread in published work.
Choosing the right distribution
| Statistic | Use it when | Needs df? |
|---|---|---|
| Z | Population SD known, or a large sample | No |
| t | Population SD estimated from the sample | Yes |
| χ² | Categorical counts — goodness of fit or independence | Yes |
| r | Testing whether a correlation differs from zero | Yes (n − 2) |
The t and Z distributions converge as degrees of freedom grow. By df = 30 the difference in the two-tailed 5% critical value is about 0.04; by df = 100 it is roughly 0.01. Below df = 30, using Z where t is appropriate will understate your p-value and overstate your confidence.
Worked example
A two-sample t-test on 24 participants (df = 22) returns t = 2.31.
Two-tailed, that gives p = 0.031 — significant at the conventional 0.05 threshold. The two-tailed critical value at df = 22 is 2.074, and 2.31 exceeds it.
Had the same statistic come from df = 5, the critical value would be 2.571 and the result would not reach significance, with p = 0.069. The same t, a different verdict, purely because of sample size.
One-tailed or two-tailed
Use two-tailed unless you committed in advance to caring about only one direction. The temptation is obvious: a one-tailed test halves the p-value, so a result at p = 0.08 becomes p = 0.04. Making that switch after seeing which way the data fell is a textbook form of p-hacking, and it inflates the false positive rate to exactly the degree it appears to help.
A genuine one-tailed case is rare. It requires that an effect in the opposite direction would be treated identically to no effect at all — that you would take the same action either way. If a large effect in the unexpected direction would interest you, the test is two-tailed.
Descriptive statistics: which numbers to report
The descriptive mode of this tool returns n, mean, median, mode, standard deviation, variance, quartiles, IQR and standard error. Which of them you should actually report depends on the shape of your data:
- Roughly symmetric — mean and standard deviation.
- Skewed, or containing outliers — median and interquartile range. Both are resistant to extreme values, whereas a single outlier can move the mean and inflate the SD substantially.
- Describing the precision of your estimate — standard error, which is SD ÷ √n. This is not interchangeable with the standard deviation: the SD describes the spread of your data, the SE describes the uncertainty in your mean. Reporting SE where readers expect SD makes variability look about √n times smaller than it is.
The quartiles here use the linear interpolation method (equivalent to R's type 7 and to Excel's QUARTILE.INC). Other conventions exist — Minitab and some textbooks use a different rule — so quartile values can differ slightly between packages on small samples. This matters mainly when n is under about 20.
Common mistakes
- Treating p = 0.049 and p = 0.051 as categorically different. They are near-identical evidence. The 0.05 line is a convention, not a discovery threshold.
- Reporting "p = 0.000". A p-value is never exactly zero. Report
p < .001, which is what this tool displays. - Running many tests and reporting the significant ones. At α = 0.05, twenty independent tests on pure noise will on average produce one "significant" result. If you are testing multiple hypotheses, correct for it.
- Concluding "no effect" from a non-significant result. Absence of evidence is not evidence of absence, particularly in a small study. Check what effect size your design could actually have detected using the power analysis tool.
The method used here
The normal CDF uses the Abramowitz–Stegun approximation. The t and F distributions are computed via the regularised incomplete beta function, and χ² via the regularised lower incomplete gamma function, both evaluated with the standard continued-fraction and series expansions (Numerical Recipes) selected by argument size for numerical stability. Convergence is capped at 500 iterations with a relative tolerance of 10⁻¹⁴.
Sources
- Wasserstein, R. L. & Lazar, N. A. (2016). The ASA statement on p-values. The American Statistician, 70(2), 129–133.
- Press, W. H. et al. (2007). Numerical Recipes (3rd ed.), §6.2 and §6.4.
- Hyndman, R. J. & Fan, Y. (1996). Sample quantiles in statistical packages. The American Statistician, 50(4), 361–365.