Open In App

Role of Log Odds in Logistic Regression

Improve
Improve
Like Article
Like
Save
Share
Report

Prerequisite : Log Odds, Logistic Regression 

NOTE: It is advised to go through the prerequisite topics to have a clear understanding of this article. 

Log odds play an important role in logistic regression as it converts the LR model from probability based to a likelihood based model. Both probability and log odds have their own set of properties, however log odds makes interpreting the output easier. Thus, using log odds is slightly more advantageous over probability. 

Before getting into the details of logistic regression, let us briefly understand what odds are.

Odds : Simply put, odds are the chances of success divided by the chances of failure. It is represented in the form of a ratio. (As shown in equation given below)

Odds\ Ratio = \frac{p}{1-p}

where,
p -> success odds
1-p -> failure odds

Logistic Regression with Log odds

Now, let us get into the math behind involvement of log odds in logistic regression. In logistic regression, the odds of independent variable corresponding to a success is given by:

p = \frac{e^{\beta_0 + \beta_1 x}}{1+e^{\beta_0 + \beta_1 x}}

where, 
p -> odds of success
β0, β1 -> assigned weights 
x -> independent variable

So, the odds of failure in this case will be given by:

1-p = 1- \frac{e^{(\beta_0 + \beta_1 x)}}{1+e^{(\beta_0 + \beta_1 x)}} = \frac{1}{1+e^{(\beta_0 + \beta_1 x)}}

Therefore, the odds ratio is defined as:

\frac{p}{1-p} = \frac{\frac{e^{\beta_0 + \beta_1 x}}{1+e^{\beta_0 + \beta_1 x}}}{\frac{1}{1+e^{(\beta_0 + \beta_1 x)}}} = e^{(\beta_0 + \beta_1 x)}

Now, as discussed in the log odds article, we take the log of the odds ratio to get symmetricity in the results. Therefore, taking log on both sides gives:

\ln(\frac{p}{1-p}) = \beta_0 + \beta_1 x

which is the general equation of logistic regression. Now, in the logistic model, L.H.S contains the log of odds ratio that is given by the R.H.S involving a linear combination of weights and independent variables. 

Graphical Intuition 

i. Problem with Probability based output in Logistic Regression

Let us consider an example. Say, we build a logistic regression model to determine the probability of a person suffering from diabetes based on their sugar level. The plot for this would look like: (See Fig 1)

Fig 1 : LR model plot

The problem remains that the output of the model is only binary based on the above plot. To tackle this problem, we use the concept of log odds present in logistic regression. 

ii. Solution: Transforming Output

To solve the above discussed problem, we convert the probability-based output to log odds based output. (As shown in equation given below)

P(diabetes) \rightarrow log (odds\ of\ diabetes) =\log_e(\frac{p}{1-p})

Let us assume random values of p and see how the y-axis is transformed. 

a. Boundary values

At\ (p = 1) \\ \log_e(\frac{p}{1-p}) = \log_e(1/0) \\ = \log_e(1) - \log_e(0) \\ = 0 - (-\infin) \\ = \infin

At\ (p = 0) \\ \log_e(\frac{p}{1-p}) = \log_e(0/1) \\ = \log_e(0) - \log_e(1) \\ = (-\infin) - 0 \\ = -\infin

So, the domain of y axis is: (-∞, ∞)

b. Middle value

At\ (p = 0.5) \\ \log_e(\frac{p}{1-p}) = \log_e(0.5/0.5) \\ = \log_e(1)\\ = 0

So, at p = 0.5 -> log (odds) = y = 0.

c. At random values 

At\ (p = 0.75) \\ \log_e(\frac{p}{1-p}) = \log_e(0.75/0.25) \\ = \log_e(3)\\ = 1.09    

At\ (p = 0.3) \\ \log_e(\frac{p}{1-p}) = \log_e(0.3/0.7) \\ = \log_e(3) - \log_e(7)\\  \\ = -0.84

So, at p > 0.5 -> we get value of log(odds) in range (0, ∞)
and at p < 0.5 -> we get value of log(odds) in range (-∞, 0)

If we map these values onto a transformed plot, it would look like: (As shown in Fig 2)

Fig 2 : Transformed LR plot

Based on the value of slope (m) and intercept (c), we can easily interpret the model and get non-binary deterministic output. This is power of log odds in Logistic Regression. 

Log odds commonly known as Logit function is used in Logistic Regression models when we are looking non-binary output. This is how logistic regression is able to work as both a regression as well as classification model. For any doubt/query, comment below. 



Last Updated : 22 Oct, 2021
Like Article
Save Article
Previous
Next
Share your thoughts in the comments
Similar Reads