Related Articles
Difference between Gradient descent and Normal equation
• Difficulty Level : Basic
• Last Updated : 08 May, 2021

Gradient descent is an optimization algorithm used to find the values of parameters of a function that minimizes a cost function. It is an iterative algorithm. We use gradient descent to update the parameters of the model. Parameters refer to coefficients in Linear Regression and weights in neural networks.

Gradient descent can also converge even if the learning rate is kept fixed.

Normal Equation :
Normal Equation, an analytical approach used for optimization. It is an alternative for Gradient descent. Normal equation performs minimization without iteration.
Normal equations are equations obtained by setting equal to zero the partial derivatives of the sum of squared errors or cost function; normal equations allow one to estimate the parameters of multiple linear regression.

Where
X = input features value
y = output value

If the term X is non-invertible or singular then we can use regularization.

Difference between Gradient Descent and Normal Equation.