How can a result be statistically significant but practically small?

Because significance and practical importance answer different questions. A significance test assesses sample evidence against a null hypothesis, often one that says there is no population difference. It does not measure the size or usefulness of the estimated difference. [S1]

Imagine a fictional study comparing two ways of practicing a skill. The first group averages 72.0 points on a quiz and the second averages 71.2: a difference of 0.8 points. Suppose the analysis reports p = 0.01 and a 95% confidence interval for the difference of 0.2 to 1.4 points. These are illustrative assumptions, not real study findings.

The result may be called statistically significant under a chosen threshold, but the estimated gain is still less than two points. Whether that matters depends on the setting and what change would be worthwhile. The p-value is considered in relation to the null hypothesis; it is not a score for how important the result is. [S2]

Treat it as an interval estimate for the unknown difference, not as a guarantee of the exact effect. A confidence interval estimates an unknown population parameter, and its width depends on factors including the confidence level and sample size. [S2]

In the fictional example, the interval from 0.2 to 1.4 points conveys uncertainty around the estimated 0.8-point difference. Its lower end is still a small change; its upper end is larger, but may also be modest for the decision at hand. The interval does not tell you by itself whether either value is practically worthwhile. That requires a meaningful benchmark for this particular situation. The cited definitions do not establish a rule for deciding practical importance.

Set the practical benchmark using the purpose of the measurement, not the significance label. Ask what improvement would be noticeable or useful to the people making the decision, and whether the time, effort, or other trade-offs are worth that improvement. For a quiz, a teacher might ask whether a fraction of a point changes instruction or learning decisions; that judgment depends on the context, not on the p-value.

For example, if the team had decided beforehand that a change under 2 points would not affect its decision, then the example’s estimated 0.8-point gain—and even the interval’s upper end of 1.4—would remain below that benchmark. That would support describing the result as statistically significant but practically small for that stated purpose. The 2-point benchmark here is a fictional assumption, not a general standard.

Yes. Report the estimated difference and its uncertainty first, then describe the significance result and explain the practical benchmark. Keep the conclusion limited to what was measured; do not turn a small difference into a claim of broad benefit.

Copyable example, using only the fictional figures above:

“Group A’s average was 0.8 points higher than Group B’s (95% confidence interval: 0.2 to 1.4; p = 0.01). This is evidence against a no-difference null hypothesis under the analysis used. However, if changes under 2 points would not alter our decision, the estimated effect is practically small for this purpose. The interval and test do not establish whether the difference would matter in another setting.”

Replace the figures and benchmark with the actual analysis and a context-appropriate judgment. A significance result alone does not establish that an effect is useful or important.