Univariate data is the type of data in which the result depends only on one variable. For instance, dataset of points on a line can be considered as a univariate data where abscissa can be considered as input feature and ordinate can be considered as output/result.
For line Y = 2X + 3;
Input feature will be X and Y will be the result.
Cost function computes the error percentage with the current value of regression coefficients. It quantitatively defines how far the model is wrt actual regression coefficients which has lowest rate of error.
After each iteration, the cost is upgraded in proportion to the error. The nature of error is very data sensitive. By data sensitive i mean the error value changes very fast, because we had square in error function. Hence, to make it more tolerant to high values of errors, we derivate the error function.
The mathematics is as follows:
At each iteration (epoch), the values of the regression coefficient are updated by a specific value wrt to the error from the previous iteration. This updation is very crucial and is the crux of the machine learning applications that you write.
Updating the coefficients with exact an update of a coefficient is done by penalizing its value with a fraction of error that it’s previous values caused.
This fraction is called as learning rate. This defines how fast our model reaches out to point of convergence(point where error is ideally 0).
Python function for the same is as follows:
This is the function which is used to specify when should the iterations should stop.
As per user, the algorithm stop_iteration generally returns true in following conditions:
- Max Iteration : Model is trained for a specified number of iterations.
- Error value : Depending upon the value of previous error, the algorithm decides whether to continue or stop.
- Accuracy : Depending upon the last accuracy of the model, if it is larger than the mentioned accuracy, the algorithm returns True,
- Hybrid : This is more often used. This combines more than one above mentioned conditions along with an exceptional break option. Exceptional break is condition where training continues until when something bad happens. Something bad might include overflow of result, time constraints exceeded, etc.
Having all the utility functions defined, lets see the pseudo code followed by its implementation:
Final Oop implementation :
- Linear Regression (Python Implementation)
- Python | Linear Regression using sklearn
- ML | Linear Regression
- Linear Regression using PyTorch
- Linear Regression Using Tensorflow
- Gradient Descent in Linear Regression
- ML | Rainfall prediction using Linear regression
- ML | Normal Equation in Linear Regression
- Pyspark | Linear regression using Apache MLlib
- Mathematical explanation for Linear Regression working
- ML | Boston Housing Kaggle Challenge with Linear Regression
- Pyspark | Linear regression with Advanced Feature Dataset using Apache MLlib
- ML | Logistic Regression using Python
- Random Forest Regression in Python
- Python | Implementation of Polynomial Regression
If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to email@example.com. See your article appearing on the GeeksforGeeks main page and help other Geeks.
Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.