In essence, a claim is assumed valid if its counter-p value table for t test pdf is improbable. A result is said to be statistically significant if it allows us to reject the null hypothesis.
That is, as per the reductio ad absurdum reasoning, the statistically significant result should be highly improbable if the null hypothesis is assumed to be true. The rejection of the null hypothesis implies that the correct hypothesis lies in the logical complement of the null hypothesis. However, unless there is a single alternative to the null hypothesis, the rejection of null hypothesis does not tell us which of the alternatives might be the correct one. However, supposing we manage to reject the zero mean hypothesis, even if we know the distribution is normal and variance is unity, the null hypothesis test does not tell us which non-zero value we should adopt as the new mean.
Thus, this naive definition is inadequate and needs to be changed so as to accommodate the continuous random variables. However, that does not prove that the tested hypothesis is true. This statistic provides a single number, such as the average or the correlation coefficient, that summarizes the characteristics of the data, in a way relevant to a particular inquiry. As such, the test statistic follows a distribution determined by the function used to define that test statistic and the distribution of the input observational data. For the important case in which the data are hypothesized to follow the normal distribution, depending on the nature of the test statistic and thus the underlying hypothesis of the test statistic, different null hypothesis tests have been developed. Here a few simple examples follow, each illustrating a potential pitfall.
The researcher rolls the dice and observes that both dice show 6, yielding a test statistic of 12. If the researcher assumed a significance level of 0. 05, this result would be deemed significant and the hypothesis that the dice are fair would be rejected. Suppose a researcher flips a coin five times in a row and assumes a null hypothesis that the coin is fair. The test statistic of “total number of heads” can be one-tailed or two-tailed: a one-tailed test corresponds to seeing if the coin is biased towards heads, but a two-tailed test corresponds to seeing if the coin is biased either way. 05, this result would be deemed significant and the hypothesis that the coin is fair would be rejected.
06, which is not significant at the 0. The test statistic is the total number of heads and is a two-tailed test. 0625, which is not significant at the 0. 002, which is significant at the 0.