Residual Sum of Squares

Last Updated : 30 Jun, 2023

Residual Sum of Squares is one of the types of sum of squares in regression which is used to measure the dispersion of the data points. The sum of squares can also be used to calculate the variance in the values of assets in the case of accounting. If the value of the sum of squares is higher, it represents a higher variance from the mean value and vice versa. The sum of squares is generally of 3 types i.e. Total Sum of Squares, Regressive or Regression Sum of Squares, and Residual Sum of Squares. In this article, we will study majorly the types of Residual Sum of Squares. Other than this we will also discuss both other types in bried as well.

What is Residual Sum of Squares (RSS)?

Residual sum of squares is used to calculate the variance of the data in terms of error or residuals. It is used to calculate the error left between regression data and regression function after running the model. The smaller the value of the residual sum of squares, the better the model.

Residual Sum of Square Formula

Residual Sum of Squares (RSS) can be calculated using the following formula:

$\bold{RSS= \Sigma_{i=1}^n(y_i-f(x_i))^2}$

Where,

y_i is the i^thvalue of variable to be predicted,
f(x_i) is the predicted value, and
n is the number of terms or variables.

Regression Sum of Squares (SSR)

The regression sum of squares measures how well the model is and how close is the predicted value to the expected value.

Regression Sum of Squares Formula

Consider a set X with n observations. The sum of squares S for this set can be calculated using the below formula:

$\bold{S = \Sigma_{i=1}^{n} (X_i- \bar{X})^2}$

Where,

X_i is the i^th observation of the set,
$\bold{\bar{X}}$ is the mean of the dataset, and
n is the number of observations.

Total Sum of Squares (TSS)

Total sum of squares is used to denote the amount of variation in the dependent variable. The total sum of squares is the sum of the regression sum of squares and the residual sum of squares. It is calculated as:

TSS = RSS + SSR

Where the abbreviations have their usual meaning.

How to Calculate Sum of Squares?

We will discuss steps to calculate the sum of squares for both the residual method and regressive method in the following headings.

How to Calculate Residual Sum of Squares?

To calculate the residual sum of squares, we can use the following steps:

Step 1: Organize the data to find the expected value.

Step 2: Calculate the residual i.e., y_i – ŷ_i.

Step 3: Use the following formula to calculate the Residual Sum of Squares.

$\bold{RSS= \Sigma_{i=1}^n(y_i-f(x_i))^2}$

Step 4: The result is the required value of the Residual Sum of Squares.

How to Calculate Sum of Squares Due to Regression?

To calculate the sum of squares due to regression we can use the following steps:

Step 1: Calculate the mean of the given data
Step 2: Calculate the difference between the mean and each data point.
Step 3: Calculate the square of the value obtained in step 2.
Step 4: Sum all the values obtained from Step 3.

Learn more about Sum of Squares

Significance and Limitations

Significance of Sum of Squares

The sum of squares formula can be used for various purposes and has great significance in real life such as:

It can be used to find the variability of data points from the mean value.
It helps the investors to make a good decision regarding investment by checking the variance of the stock.
It can also help to compare the stock price of two different companies.

Limitations of Sum of Squares

The sum of squares has the following limitations:

A higher dataset makes it very difficult to make decisions in real life as the graph of the data is more spread out.
An investor may need data of many years to make good decisions but this huge data becomes very difficult to handle.

Solved Examples of Residual Sum of Squares

Problem 1: Calculate the sum of squares of the set X = [1,2,3,6] if the mean is found to be 3.

Solution:

Given $\bar{X} = 3$

X

$X-\bar{X}$

1

-2

2

-1

3

0

6

3

Using $S = \Sigma_{i=1}^{n} (X_i- \bar{X})^2$

$S = (-2)^2+(-1)^2+0^2+3^2$

S = 4+1+0+9

S = 14

Therefore , The sum of squares of the set is 14.

X	$X-\bar{X}$
1	-2
2	-1
3	0
6	3

Problem 2: Calculate the sum of squares of the set X = [3,6,9,12,15] if the mean is found to be 9.

Solution:

Given $\bar{X} = 9$

X

$X-\bar{X}$

3

-6

6

-3

9

0

12

3

15

6

Using $S = \Sigma_{i=1}^{n} (X_i- \bar{X})^2$

$S = (-6)^2+(-3)^2+0^2+3^2+6^2$

S = 36+9+0+9+36

S = 90

$\therefore$ The sum of squares of the set is 90.

X	$X-\bar{X}$
3	-6
6	-3
9	0
12	3
15	6

Problem 3: Calculate the sum of squares of the dataset X = [1,2,3,4,5,6]

Solution:

In this case we need to calculate the mean first.

$\bar{X} = \frac{1+2+3+4+5+6}{6}$

= 21/6

$\bar {X} = 3.5$

X

$X-\bar{X}$

1

-2.5

2

-1.5

3

-0.5

4

0.5

5

1.5

6

2.5

Using $S = \Sigma_{i=1}^{n} (X_i- \bar{X})^2$

$S = (-2.5)^2+(-1.5)^2+(-0.5)^2+(0.5)^2+(1.5)^2+(2.5)^2$

S = 6.25+2.25+0.25+0.25+2.25+6.25

S = 17.50

$\therefore$ The sum of squares of the set is 17.50.

X	$X-\bar{X}$
1	-2.5
2	-1.5
3	-0.5
4	0.5
5	1.5
6	2.5

Problem 4: Calculate the sum of squares of the dataset Y = [3,4,5,1,7]

Solution:

In this case we need to calculate the mean first.

$\bar{X} = \frac{3+4+5+1+7}{5}$

= 20/5

$\bar {X} = 4$

X

$X-\bar{X}$

3

-1

4

0

5

1

1

-3

7

3

Using $S = \Sigma_{i=1}^{n} (X_i- \bar{X})^2$

$S = (-1)^2+(0)^2+(1)^2+(-3)^2+(3)^2$

S = 1+0+1+9+9

S = 20

$\therefore$ The sum of squares of the set is 20.

X	$X-\bar{X}$
3	-1
4	0
5	1
1	-3
7	3

Problem 5: Calculate the sum of squares of the set X = [1,4,6,8] if mean is found to be 4.75.

Solution:

Given $\bar{X} = 4.75$

X

$X-\bar{X}$

1

-3.75

4

-0.75

6

1.25

8

3.25

Using $S = \Sigma_{i=1}^{n} (X_i- \bar{X})^2$

$S = (-3.75)^2+(-0.75)^2+(1.25)^2+(3.25)^2$

S = 14.0625+0.5625+1.5625+10.5625

S = 26.75

$\therefore$ The sum of squares of the set is 26.75.

X	$X-\bar{X}$
1	-3.75
4	-0.75
6	1.25
8	3.25

FAQs Of Residual Sum of Squares

Q1: What is the Practical Significance of Sum of Squares?

Answer:

Sum of squares helps investors to determine the variability in the price of the stock and helps them to invest their money wisely.

Q2: What is the Desirable Value for Residual Sum of Squares?

Answer:

We always desire a low value of residual sum of squares as it is calculated by considering the error left after the model is run. So a lower value will mean lesser error as compared to a higher value.

Q3: What is Regression?

Answer:

Regression is a technique in statistics that determines the relationship between a dependent variable and one or more independent variables. It is generally used for prediction purposes.

Q4: What are The Types of Sum of Squares used in Regression?

Answer:

Sum of squares are classified into 3 types which are:

Regression Sum of Squares
Residual Sum of Squares
Total Sum of Squares

Suggest improvement

Sum of Squares

Share your thoughts in the comments

Residual Sum of Squares

What is Residual Sum of Squares (RSS)?

Residual Sum of Square Formula

Regression Sum of Squares (SSR)

Regression Sum of Squares Formula

Total Sum of Squares (TSS)

How to Calculate Sum of Squares?

How to Calculate Residual Sum of Squares?

How to Calculate Sum of Squares Due to Regression?

Significance and Limitations

Significance of Sum of Squares

Limitations of Sum of Squares

Solved Examples of Residual Sum of Squares

FAQs Of Residual Sum of Squares

Q1: What is the Practical Significance of Sum of Squares?

Q2: What is the Desirable Value for Residual Sum of Squares?

Q3: What is Regression?

Q4: What are The Types of Sum of Squares used in Regression?

Please Login to comment...

Similar Reads

What kind of Experience do you want to share?