How to Perform McNemar’s Test in R

McNemar’s test is a statistical method used to analyze paired categorical data, often applied when comparing two related groups. It helps to determine if there is a significant difference in proportions or frequencies between the two groups. This test is particularly useful when the data is not normally distributed or when analyzing data from before-and-after studies, treatment-control studies, or matched-pair designs.

McNemar’s Test

McNemar’s test can be easily performed in R Programming Language using built-in functions. The test requires a contingency table with counts of paired observations falling into four categories: agreement on both measurements, disagreement on both measurements, agreement on the first but not the second, and agreement on the second but not the first.

Syntax

mcnemar.test(x, y = NULL, correct = TRUE)

• x: either a two-dimensional contingency table in matrix form, or a factor object.
• y: a factor object; ignored if x is a matrix.
• correct: TRUE = apply continuity correction when computing test statistic; FALSE = do not apply continuity correction.

Create the Data to perform McNemar’s Test in R

R

# Create a 2x2 matrix representing the contingency table
data <- matrix(c(30, 12, 20, 25), nrow = 2,
               dimnames = list("After Method B" = c("Passed", "Failed"),
                               "After Method A" = c("Passed", "Failed")))

# Display the contingency table
print("Contingency table:")
print(data)

Output:

[1] "Contingency table:"
              After Method A
After Method B Passed Failed
        Passed     30     20
        Failed     12     25

The contingency table represents the number of students who passed and failed after being taught using Method A and Method B.

Now we will perform Mc.emar’s Test.

R

# Perform McNemar's Test with continuity correction
result_with_correction <- mcnemar.test(data)
print("McNemar's Test with continuity correction:")
print(result_with_correction)

# Perform McNemar's Test without continuity correction
result_without_correction <- mcnemar.test(data, correct = FALSE)
print("McNemar's Test without continuity correction:")
print(result_without_correction)

Output:

[1] "McNemar's Test with continuity correction:"

    McNemar's Chi-squared test with continuity correction

data:  data
McNemar's chi-squared = 1.5312, df = 1, p-value = 0.2159

[1] "McNemar's Test without continuity correction:"

    McNemar's Chi-squared test

data:  data
McNemar's chi-squared = 2, df = 1, p-value = 0.1573

McNemar’s test with continuity correction to the data. The result includes McNemar’s chi-squared statistic, degrees of freedom, and the p-value.

• The code also performs McNemar’s test without continuity correction, yielding the chi-squared statistic, degrees of freedom, and the p-value.
• Both tests help assess whether there’s a significant difference in outcomes between Method A and Method B. In this example, the p-values suggest that there’s no significant difference between the two methods, as they are both above the conventional significance level of 0.05.

Perform McNemar’s Test on a Medical Data

R

# Create a 2x2 contingency table representing the counts of symptom relief outcomes
data <- matrix(c(70, 30, 60, 40), nrow = 2,
               dimnames = list("After Treatment B" = c("Relieved", "Not Relieved"),
                               "After Treatment A" = c("Relieved", "Not Relieved")))

# Display the contingency table
print("Contingency table:")
print(data)

# Perform McNemar's Test with continuity correction
result_with_correction <- mcnemar.test(data)
print("McNemar's Test with continuity correction:")
print(result_with_correction)

# Perform McNemar's Test without continuity correction
result_without_correction <- mcnemar.test(data, correct = FALSE)
print("McNemar's Test without continuity correction:")
print(result_without_correction)

Output:

[1] "Contingency table:"
                 After Treatment A
After Treatment B Relieved Not Relieved
     Relieved           70           60
     Not Relieved       30           40

[1] "McNemar's Test with continuity correction:"

    McNemar's Chi-squared test with continuity correction

data:  data
McNemar's chi-squared = 9.3444, df = 1, p-value = 0.002237

[1] "McNemar's Test without continuity correction:"

    McNemar's Chi-squared test

data:  data
McNemar's chi-squared = 10, df = 1, p-value = 0.001565

The contingency table compares the outcomes of Treatment A and Treatment B in terms of symptom relief.

• We have counts for “Relieved” and “Not Relieved” patients for both Treatment A and Treatment B.
• McNemar’s test is performed with and without continuity correction to assess if there’s a statistically significant difference in the effectiveness of the two treatments in providing symptom relief.
• The test statistic (McNemar’s chi-squared) is calculated to be approximately 9.3444.
• The degrees of freedom (df) is 1.
• The p-value associated with this test is 0.002237, indicating that there is strong evidence to reject the null hypothesis that the two treatments have the same effect on symptom relief, in favor of the alternative hypothesis that they differ.
• The test statistic (McNemar’s chi-squared) is calculated to be 10.
• The degrees of freedom (df) is 1.
• The p-value associated with this test is 0.001565, again indicating strong evidence to reject the null hypothesis in favor of the alternative hypothesis.

Both tests suggest that there’s a significant difference in the effectiveness of Treatment A and Treatment B in providing symptom relief, with Treatment B appearing to be more effective based on the given data.

Advantages

1. McNemar’s test is specifically designed to analyze data where each observation falls into one of four categories, making it ideal for paired data comparisons.
2. Unlike some other statistical tests, McNemar’s test does not assume a normal distribution of data, making it robust for non-normally distributed data.
3. The test yields a single p-value, allowing researchers to easily determine whether there’s a significant difference between the paired groups.

Limitations

1. McNemar’s test assumes that observations are independent within each group, which may not always hold true in practice.
2. The test may be less reliable with small sample sizes, leading to inaccurate results or low statistical power.
3. It’s best suited for binary outcome variables, limiting its use with other types of categorical data.
4. The test may not perform well if there is significant asymmetry between the two groups being compared.

Conclusion

McNemar’s test is a powerful statistical tool for analyzing paired categorical data, commonly used in before-and-after studies, treatment-control experiments, and matched-pair designs. It provides insights into whether there is a significant difference in proportions or frequencies between two related groups. While McNemar’s test has its advantages, such as simplicity and applicability to small sample sizes, it also has limitations, including the assumption of marginal homogeneity and sensitivity to rare events. Understanding these aspects is crucial for accurate interpretation of results and informed decision-making in various research contexts.