p-value

A p-value is not the probability that your hypothesis is true, nor that the result happened by chance — it is a statement about how surprising your data would be in a world where no effect exists.

Modern reporting standards pair every p-value with an effect size and, where possible, a confidence interval, because significance without magnitude says little.

The 0.05 threshold is a convention, not a law of nature. It was proposed as a rough guide and hardened into a rule, which is why a result at p = 0.049 is often treated as meaningfully different from one at p = 0.051 when the evidence is practically identical. Report exact p-values (p = 0.032, not p < 0.05) so readers can judge the strength of evidence themselves, and reserve 'p < 0.001' for genuinely very small values.

Two practical traps recur in theses. First, sample size drives significance: with 1,000 respondents almost any difference becomes significant, while with 40 a substantial effect may not. Second, running many tests inflates the chance of false positives — twenty independent tests at 0.05 will on average produce one 'significant' result by chance alone. Where you test many hypotheses or compare many groups, apply a correction (Bonferroni, Holm or a post-hoc procedure) and say so in your methods.