# Generalized Linear Models

**Prerequisite:**

The following article discusses the **Generalised linear models **(GLMs) which explains how Linear regression and Logistic regression are a member of a much broader class of models. GLMs can be used to construct the models for regression and classification problems by using the type of distribution which best describes the data or labels given for training the model. Below given are some types of datasets and the corresponding distributions which would help us in constructing the model for a particular type of data (The term data specified here refers to the **output data** or the **labels** of the dataset).

Attention reader! Don’t stop learning now. Get hold of all the important Machine Learning Concepts with the **Machine Learning Foundation Course** at a student-friendly price and become industry ready.

- Binary classification data – Bernoulli distribution
- Real valued data – Gaussian distribution
- Count-data – Poissons distribution

To understand GLMs we will begin by defining exponential families. Exponential families are a class of distributions whose probability density function(PDF) can be molded into the following form:

**Proof** – **Bernoulli distribution** is a member of the exponential family.

Therefore, on comparing

**Eq1**and

**Eq2**:

Note:As mentioned above the value ofphi(which is the same as the activation orsigmoid functionfor Logistic regression) is not a coincidence. And it will be proved later in the article how Logistic regression model can be derived from the Bernoulli distribution.

**Proof **– **Gaussian distribution** is a member of the exponential family.

Therefore, on comparing

**Eq1**and

**Eq3**:

**Constructing GLMs:**

To construct GLMs for a particular type of data or more generally for linear or logistic classification problems the following three assumptions or the design choices are to be considered:

The first assumption is that if

**x**is the input data parameterized by theta the resulting output or y will be a member of the exponential family. This means if we are provided with some labeled data our goal is to find the right parameters theta which fits the given model as closely as possible. The third assumption is the least justified and can be considered as a design choice.

**Linear Regression Model:**

To show that Linear Regression is a special case of the GLMs. It is considered that the output labels are **continuous values** and are therefore a **Gaussian distribution**. So, we have

The first equation above corresponds to the first assumption that the output labels (or

**target variables**) should be the member of an exponential family, Second equation corresponds to the assumption that the

**hypothesis**is equal the

**expected value**or

**mean**of the distribution and lastly, the third equation corresponds to the assumption that natural parameter and the input parameters follow a linear relationship.

**Logistic Regression Model:**

To show that Logistic Regression is a special case of the GLMs. It is considered that the output labels are **Binary valued** and are therefore a **Bernoulli distribution**. So, we have

From the third assumption, it is proven that:

The function that maps the natural parameter to the canonical parameter is known as the

**canonical response function**(here, the log-partition function) and the inverse of it is known as the

**canonical link function**.

Therefore by using the three assumptions mentioned before it can be proved that the

**Logistic**and

**Linear Regression**belongs to a much larger family of models known as GLMs.

Reference: http://cs229.stanford.edu/notes/cs229-notes1.pdf