Testing research hypotheses: how to draw conclusions from statistical tests
The verification of the hypothesis does not consist in checking whether the p-value is below the accepted threshold; it is necessary to separate the research hypothesis from the statistical one, to correctly describe the result and to evaluate its relevance in the context of the method, test and previous studies.
Key information
- ✓Write down the research hypothesis and its corresponding statistical hypothesis.
- ✓Interpret the p-value according to the accepted analysis plan.
- ✓Report the magnitude of the effect and the confidence intervals, when appropriate.
- ✓Do not change the hypothesis after viewing the data without clearly labeling the exploratory analysis.
Research and statistical hypothesis
A research hypothesis describes an expected dependence or difference in a material language. A null hypothesis and an alternative hypothesis translate it into a statistical model.
What does the p-value say?
The p-value describes the conformity of the data to the assumed zero model, not the likelihood of the hypothesis being true. Its interpretation depends on the accepted level of relevance and test conditions.
Effect and Uncertainty
Two analyses can have the same p-value, but very different effects, so when the method allows for this, give the measure of effect and the confidence interval, so they show the scale of the phenomenon and the accuracy of the estimate.
The final sentence of the chapter
First, give the test result, then the statistical hypothesis decision, and finally the substantive conclusion. Avoid formulations to prove the hypothesis.
Summary
The correct conclusion combines the test result with the hypothesis, the magnitude of the effect and the limitations of the study.
Frequently asked questions
Does a trivial result mean that the hypothesis is false?
No. It means that there is not enough evidence to reject the null hypothesis in the design and data in question. The significance of the result also depends on the power of the study and the accuracy of the measurement.
Need support with your work?
Describe the topic, scope and deadline. We will review the materials, recommend the right support and prepare a clear quote.
