Find the Regression Output in R

In R Programming Language we can Interpret Regression Output by using various functions depending on the type of regression analysis you are conducting. The two most common types of regression analysis are linear regression and logistic regression. Here, I’ll provide examples of how to find the regression output for both types.

Linear regression is used to understand and model the relationship between a dependent variable (Y) and one or more independent variables (X1, X2, etc.).

The goal is to find the best-fitting linear equation that describes how changes in the independent variables are associated with changes in the dependent variable.

Linear Equation

In simple linear regression, with one independent variable, the linear equation takes the form:

Y = β0 + β1 * X + ε

• Y: Dependent variable (the variable we want to predict).
• X: Independent variable (the variable used to make predictions).
• β0: Intercept (the value of Y when X is 0).
• β1: Slope (the change in Y for a one-unit change in X).
• ε: Error term (represents the unexplained variation or random noise).

Parameters Estimation

The goal is to estimate the values of β0 and β1 that minimize the sum of squared differences between the observed values of Y and the predicted values of Y (the “least squares” criterion).

Model Interpretation

The β1 coefficient represents the change in the dependent variable for a one-unit change in the independent variable, assuming all other variables are held constant.

The β0 intercept represents the value of the dependent variable when the independent variable is zero (sometimes it may not have a meaningful interpretation).

Model Assumptions

Linear regression assumes that the relationship between variables is linear.

Assumptions also include homoscedasticity (constant variance of errors), independence of errors, and normally distributed errors.

Model Evaluation

Model performance is often assessed using metrics like R-squared (the proportion of variance explained by the model), Mean Squared Error (MSE), or Root Mean Squared Error (RMSE).

Residual analysis (plotting the differences between observed and predicted values) helps identify potential issues.

Types of Linear Regression

1. Simple Linear Regression: Involves one independent variable.
2. Multiple Linear Regression: Involves two or more independent variables.
3. Polynomial Regression: Uses polynomial equations to fit curves rather than straight lines.

Linear regression is used in various fields for tasks such as predicting sales based on advertising expenditure, estimating house prices based on features, and understanding the relationship between variables in scientific research.

Implementation in R

In R, you can use the lm() function to fit linear regression models. The summary() function provides detailed output with coefficients, p-values, and goodness-of-fit statistics.

Linear Regression Output

For linear regression, you typically use the lm() function to fit a linear model, and then you can use the summary() function to obtain detailed regression output. Here’s a step-by-step guide:

Fit the Linear Model:

Use the lm() function to fit a linear regression model. For example:

# Create a simple linear regression model
model <- lm(Y ~ X1 + X2, data = your_data)

Replace Y, X1, X2, and our_data with your specific response variable, predictor variables, and data.

View Regression Summary:

To view the regression summary, use the summary() function on the fitted model:

# View the regression summary
summary(model)

This will display detailed information about the linear regression model, including coefficients, p-values, R-squared, and more.

Logistic Regression Output:

For logistic regression, you use the glm() function (generalized linear model) to fit the model, and you can use the summary() function as well. Here’s how to do it:

Fit the Logistic Model:

Use the glm() function to fit a logistic regression model. For example:

# Create a logistic regression model
model <- glm(Y ~ X1 + X2, data = your_data, family = binomial)

Replace Y, X1, X2, your_data, and binomial with your specific response variable, predictor variables, data, and family distribution (e.g., binomial for logistic regression).

View Regression Summary:

To view the regression summary, use the summary() function on the fitted model:

# View the regression summary
summary(model)

This will display detailed information about the logistic regression model, including coefficients, p-values, and goodness-of-fit statistics.

Interpret Regression Output for Simple Linear Regression

R

# Create a sample dataframe for linear and logistic regression
set.seed(123)
# number of samples
n <- 100

# Linear regression variables
Advertising <- rnorm(n, mean = 50, sd = 10)
Price <- rnorm(n, mean = 100, sd = 20)
Sales <- 30 + 2 * Advertising - 3 * Price + rnorm(n, mean = 0, sd = 5)

# Logistic regression variables
Age <- rnorm(n, mean = 40, sd = 10)
Gender <- sample(c("Male", "Female"), n, replace = TRUE)
Outcome <- rbinom(n, size = 1, prob = 0.7)

# Create the dataframe
sample_data <- data.frame(Advertising, Price, Sales, Age, Gender, Outcome)
head(sample_data)

Output:

  Advertising     Price     Sales      Age Gender Outcome
1    44.39524  85.79187 -127.5911 32.84758   Male       0
2    47.69823 105.13767 -183.4545 32.47311   Male       1
3    65.58708  95.06616 -125.3500 30.61461 Female       0
4    50.70508  93.04915 -145.0213 29.47487 Female       1
5    51.29288  80.96763 -112.3888 35.62840   Male       1
6    67.15065  99.09945 -135.3783 43.31179   Male       1

Create model for Interpret Regression Output

R

# Fit a linear regression model
linear_model <- lm(Sales ~ Advertising + Price, data = sample_data)

# View the regression summary
summary(linear_model)

Output:

Call:
lm(formula = Sales ~ Advertising + Price, data = sample_data)

Residuals:
    Min      1Q  Median      3Q     Max 
-9.3651 -3.3037 -0.6222  3.1068 10.3991 

Coefficients:
            Estimate Std. Error  t value Pr(>|t|)    
(Intercept) 33.40933    3.72226    8.976 2.18e-14 ***
Advertising  1.93341    0.05243   36.873  < 2e-16 ***
Price       -2.99405    0.02475 -120.978  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 4.756 on 97 degrees of freedom
Multiple R-squared:  0.9941,    Adjusted R-squared:  0.994 
F-statistic:  8239 on 2 and 97 DF,  p-value: < 2.2e-16

We use the lm() function to fit a linear regression model, where Sales is our dependent variable and Advertising and Price are independent variables.

The summary() function provides detailed output, including coefficients (intercept, Advertising, and Price), standard errors, t-values, p-values, and R-squared.

Call:
lm(formula = Sales ~ Advertising + Price, data = sample_data)

We are trying to figure out how changes in two factors, advertising and price, affect sales.
The term “Sales” is what we’re trying to predict, and “Advertising” and “Price” are the things we think might influence it.
The data we’re using for this analysis comes from a dataset named sample_data. The goal is to understand the relationships between advertising, price, and sales in the real world.

Residuals:
    Min      1Q  Median      3Q     Max 
-9.3651 -3.3037 -0.6222  3.1068 10.3991 

• Residuals represent the differences between the predicted sales and the actual sales.
• The range of these differences varies from a minimum of -9.3651 to a maximum of 10.3991.

Coefficients:
            Estimate Std. Error  t value Pr(>|t|)    
(Intercept) 33.40933    3.72226    8.976 2.18e-14 ***
Advertising  1.93341    0.05243   36.873  < 2e-16 ***
Price       -2.99405    0.02475 -120.978  < 2e-16 ***

The model estimates three coefficients:

• Intercept : The starting point for sales when both advertising and price are zero.
• Advertising : For each unit increase in advertising, sales are expected to increase by approximately 1.93 units, assuming other factors remain constant.
• Price : For each unit increase in price, sales are expected to decrease by approximately 2.99 units, assuming other factors remain constant.
• Standard errors indicate the precision of these estimates.
• The t-values and p-values assess whether these coefficients are significantly different from zero.

Residual standard error: 4.756 on 97 degrees of freedom
Multiple R-squared:  0.9941,    Adjusted R-squared:  0.994 
F-statistic:  8239 on 2 and 97 DF,  p-value: < 2.2e-16

• Residual Standard Error: This is an estimate of the average difference between predicted and actual sales, approximately 4.756 units.
• Multiple R-squared and Adjusted R-squared: These values (around 0.9941) indicate how well advertising and price collectively explain the variation in sales. The higher, the better.
• F-Statistic and p-value: The F-statistic tests whether there is a significant relationship between advertising, price, and sales. A very small p-value (less than 2.2e-16) suggests that at least one of the predictors is significant in explaining sales.

Interpret Regression Output for Logistic Regression

R

# Create a sample dataframe for linear and logistic regression
set.seed(123)
# number of samples
n <- 100

# Linear regression variables
Advertising <- rnorm(n, mean = 50, sd = 10)
Price <- rnorm(n, mean = 100, sd = 20)
Sales <- 30 + 2 * Advertising - 3 * Price + rnorm(n, mean = 0, sd = 5)

# Logistic regression variables
Age <- rnorm(n, mean = 40, sd = 10)
Gender <- sample(c("Male", "Female"), n, replace = TRUE)
Outcome <- rbinom(n, size = 1, prob = 0.7)

# Create the dataframe
sample_data <- data.frame(Advertising, Price, Sales, Age, Gender, Outcome)

head(sample_data)

Output:

  Advertising     Price     Sales      Age Gender Outcome
1    44.39524  85.79187 -127.5911 32.84758   Male       0
2    47.69823 105.13767 -183.4545 32.47311   Male       1
3    65.58708  95.06616 -125.3500 30.61461 Female       0
4    50.70508  93.04915 -145.0213 29.47487 Female       1
5    51.29288  80.96763 -112.3888 35.62840   Male       1
6    67.15065  99.09945 -135.3783 43.31179   Male       1

Create model for Interpret Regression Output

R

# Fit a logistic regression model
logistic_model <- glm(Outcome ~ Age + Gender, data = sample_data, family = binomial)

# View the regression summary
summary(logistic_model)

Output:

Call:
glm(formula = Outcome ~ Age + Gender, family = binomial, data = sample_data)

Coefficients:
            Estimate Std. Error z value Pr(>|z|)  
(Intercept)  2.26053    0.96378   2.345    0.019 *
Age         -0.02311    0.02229  -1.037    0.300  
GenderMale  -0.72437    0.47008  -1.541    0.123  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 120.43  on 99  degrees of freedom
Residual deviance: 116.49  on 97  degrees of freedom
AIC: 122.49

Number of Fisher Scoring iterations: 4

We use the glm() function for logistic regression. Here, Outcome is the binary outcome variable, and we predict it based on Age and Gender. The family = binomial argument specifies logistic regression.

• set.seed(123): This line sets a seed for the random number generator. Setting a seed ensures that you get reproducible results when generating random data. In this case, you’ve set it to 123.
• n <- 100: You’re defining the number of samples, which is 100 in this case.
• Linear Regression Variables:
• Advertising: This variable is generated using rnorm and represents some random data that follows a normal distribution with a mean of 50 and a standard deviation of 10.
• Price: Similarly, this variable is generated with a mean of 100 and a standard deviation of 20.
• Sales: This variable is generated as a linear combination of Advertising, Price, and some random noise, simulating a linear regression relationship.
• Logistic Regression Variables:
• Age: This variable represents random ages following a normal distribution with a mean of 40 and a standard deviation of 10.
• Gender: This variable is generated by randomly sampling “Male” or “Female” values for each sample with replacement.
• Outcome: This variable is generated using rbinom to simulate binary outcomes (0 or 1) with a probability of success (1) of 0.7. This is typical for logistic regression where you’re modeling a binary outcome.
• sample_data <- data.frame(…): Here, you’re creating a dataframe called sample_data by combining all the variables you generated earlier into a single dataframe. This dataframe contains columns for Advertising, Price, Sales, Age, Gender, and Outcome.
• logistic_model <- glm(Outcome ~ Age + Gender, data = sample_data, family = binomial): This line fits a logistic regression model to your sample_data. It models the binary outcome variable Outcome as a function of Age and Gender. The family = binomial argument specifies that you’re using a binomial family for logistic regression.
• summary(logistic_model): Finally, you’re viewing a summary of the logistic regression model you just fitted. The summary provides information about the coefficients, p-values, and goodness-of-fit statistics for the logistic regression model, allowing you to assess the significance of each predictor (Age and Gender) in predicting the outcome (Outcome).

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 120.43  on 99  degrees of freedom
Residual deviance: 116.49  on 97  degrees of freedom
AIC: 122.49

Number of Fisher Scoring iterations: 4

• Dispersion Parameter: This note is telling us that the statistical model being used assumes a certain level of variability in the data, and for this specific case.
• Null Deviance: The null deviance is a measure of how well our model does when we don’t consider any predictor variables. It’s like a baseline performance. Here, it’s 120.43, and we’re looking at it in comparison to the residual deviance.
• Residual Deviance: The residual deviance is how well our model performs when we include predictor variables. It’s essentially a measure of how much better our model is compared to a basic model with no predictors. In this case, it’s 116.49, suggesting our model is explaining some of the variability in the data.
• AIC (Akaike Information Criterion): The AIC is a way to balance how well our model fits the data with how complex it is. Lower AIC values are better, indicating a good fit without unnecessary complexity. In this case, the AIC is 122.49.
• Number of Fisher Scoring Iterations: This tells us how many steps the model took to find the best fit during the optimization process. In simpler terms, it’s a measure of how much effort the model had to put in to get the best results. Here, it took 4 iterations.

In both examples, the output provides information to interpret the relationship between variables, assess the significance of predictors, and understand the model’s performance. You can use these results to make predictions, conduct hypothesis tests, and draw conclusions about the relationships in your data.