Level of Significance-Definition, Steps and Examples

Level of significance or Statistical significance is an important terminology used in Statistics. Level of significance is the measurement of the statistical significance. The level of significance explains whether the null hypothesis is accepted or rejected. In this article, we are going to discuss the level of significance in detail.

What is Statistical Significance?

Research uses the term statistical significance to ascertain whether the observed difference or effect between groups is not mere chance but true. The researchers, in this way, can understand whether the study results are meaningful, or whether they could have happened just by chance.

Usually, researchers fix a level of significance (often called; α ) which denotes their cut-off point where results are considered statistically significant. The most frequently used level of significance is 0.05 implying that if the possibility of having obtained similar findings under the null hypothesis [i.e., no effect or difference] is less than 5%, then it can be said to be statistically significant.

To determine statistical significance, researchers generally engage in hypothesis testing such as t-tests, chi-square tests, ANOVA and regression analysis depending on the nature of the data and research question involved. These tests yield a p-value indicating the probability of getting these outcomes assuming the null hypothesis holds. The results are considered statistically significant when the p-value is less than those n level of significance and the null hypothesis is rejected for an alternative one instead.

What is Level of Significance?

One of the major parameters in hypothesis testing is the level of significance (denoted as the symbol α) that defines the threshold of rejecting the null hypothesis in favour of the alternative hypothesis. The level of significance represents the maximum probability of the Type I error, that is, the error of incorrectly rejecting, despite the truth of the null hypothesis.

In hypothesis testing, researchers before running the data analysis, go through the process of specifying a significance level. So the present level is an index of the assessing evidence against the null hypothesis. Frequently applied alpha values with p < 0.10, p < 0.01 or. p < 0.05, which may change as the specifics of the study provide a reason. Otherwise, researchers may set other values to find the ratio between errors of the to-type1 and type 2.

Level of Significance Symbol

The letter α (alpha) is used as the symbol to show the significance level in statistics. This is the main indicator of the accuracy of the results. This symbol annually serves as the basis of determining how to apply statistical significance.

How to Find Level of Significance?

The level of significance, which (α) is predetermined by researchers before starting a hypothesis test, can have crucial impact on the outcomes of a study. This is the risk of erroneously rejecting a true and labeled null hypothesis. In many instances, you can encounter different cut-off points for reliability, for example 0.05, 0.01, and 0.10, and researchers may choose the after the other values.

To test the level of significance, you decide before you perform your statistics test what value you want it to be. The decision in this case is usually a combination of three parameters: the nature of the research question, standards that are set within the given field of study and a desire to maintain an optimal balance between Type I and Type II error rates.

General interpretation of the p-value based upon the level of significance of 10% is added below:

• If the p-value is greater than 0.1 (i.e. p > 0.1), it means we don’t have much evidence against the null hypothesis.
• If the p-value falls between 0.05 and 0.1 (i.e. p > 0.05 and p ≤ 0.1), we have a little bit of evidence against the null hypothesis, but it’s not too strong.
• When the p-value is between 0.01 and 0.05 (i.e. p > 0.01 and p ≤ 0.05), we have stronger evidence against the null hypothesis.
• If the p-value is less than or equal to 0.01 (i.e. p ≤ 0.01), we have very strong evidence against the null hypothesis.

Level of Significance Examples

Few examples to illustrate the concept of level of significance:

Educational Research

• Researchers are looking into if a newly adopted teaching approach has a positive effect on the student’s success at the level of mathematics. These participants selected the level of significance of α = 0.10. However, following data analysis the coefficient of their study gives a p-value of 0.15 which is used to test the difference in performance of the students. Thus, as the p-value is greater than the significance level of 0.10, the researchers cannot reject its null hypothesis. Therefore, one could not claim that student performance with the two teaching methods shows the statistically different outcomes.

Medical Research

• Testing a new medication’s capacity to treat a specific condition is the mode of work for researchers in an experimental study. They will evaluate the statistical results with α = 0.05. Once the data are analyzed the associated p-value is seen to be 0.03. The fact that the calculated p-value is smaller than the specified value of p (i.e., 0.05) proves that the effect of the drug is indeed statistically significant; the researchers accept this condition, and they reject the null hypothesis; the hypothesis being that the drug is not effective.
• If p-value is equal to 0.03, then this indicates that there are 3% chances of getting a difference larger than that in our research, given that the null hypothesis exists.

Frequently Asked Questions

What is meant by Level of Significance?

Level of significance, denoted by α (alpha), is a pre-established threshold that defines the likelihood, that in actual fact the null hypothesis is right, being rejected.

What is meant by 0.05 level of significance?

Level of significance of 0.005 (i.e. p = 0.05) means that there is a 95% probability that the results found in the study are the result of a true difference between groups being compared. This also signifies that that there is a 5% chance that the results were found by chance alone and no true relationship exists between groups.

How is level of significance determined?

In most cases, researchers decide on the threshold of the statistical significance depending on the focus of the research topic, the desired level of confidence and consequences for Type I errors (false positives). In some of the studies, there are different levels of significance which can be 0.05, 0.01, 0.10, and these values depend upon the studies’ context.

What is at the significance level?

Significance level of an event (such as a statistical test) is the probability that the event could have occurred by chance.

Can level of significance be adjusted during analysis?

Level of significance is usually determined in advance as researchers conduct hypothesis testing. However, some researchers are rather flexible and are ready to adapt it if it is required under any situations. In the meantime, any other adjustment should be performed with strict justification and proper documenting.

What is role of level of significance in hypothesis testing?

Level of significance has a particular purpose of the research to determine whether the outcome is statistically significant, or an accidental outcome. This indicates that how the alternative hypothesis and null hypothesis can be used to draw conclusions from the results of statistical tests and give a rational basis for rejecting the null hypothesis.

What is level of significance in research?

Level of significance is the probability that the result reported is actually happened by chance. For example, a level of significance of 0.09 signifes that there is a 9% chance that the result is insignificant.