Moore – Penrose Pseudoinverse | Mathematics

In linear algebra pseudoinverse (A^{+}) of a matrix A is a generalization of the inverse matrix. The most common use of pseudoinverse is to compute the best fit solution to a system of linear equations which lacks a unique solution. Moore – Penrose inverse is the most widely known type of matrix pseudoinverse. The term generalized inverse is sometimes used as a synonym of pseudoinverse.

Let the system is given as:
 \overrightarrow{y} = A\overrightarrow{x}

We know A and \overrightarrow{y}, and we want to find \overrightarrow{x}.
Where:
\overrightarrow{x} and \overrightarrow{y} are vectors,
A is a matrix



If A is a square matrix, we proceed as below:
 \overrightarrow{y} = A\overrightarrow{x}

 A^{-1}\overrightarrow{y} = A^{-1}A\overrightarrow{x}

 A^{-1}\overrightarrow{y} = I\overrightarrow{x}

 A^{-1}\overrightarrow{y} = \overrightarrow{x}

But if A is not a square matrix, we cannot compute the usual A^{-1}. Nevertheless, we can form the pseudoinverse.
A^{+}\equiv pseudoinverse

If we go in detail at the system i.e. \overrightarrow{y} = A\overrightarrow{x}, then it represents the following set of equations:
 a_1_, _1 x_1 + a_1_, _2 x_2 + . . . + a_1_, _n x_n = y_1

 a_2_, _1 x_1 + a_2_, _2 x_2 + . . . + a_2_, _n x_n = y_2
. .
. .
. .
 a_m_, _1 x_1 + a_m_, _2 x_2 + . . . + a_m_, _n x_n = y_m

which can also be written in matrix form as below:
 \begin{bmatrix}  a_1_, _1 & a_1_, _2&.&.&.&a_1_, _n \\ a_2_, _1 & a_2_, _2 &.&.&.&a_2_, _n\\  . &  & & & & . \\  . &  & & & & . \\  . &  & & & & . \\ a_m_, _1 & a_m_, _2 &.&.&.&a_m_, _n\\ \end{bmatrix} % \begin{bmatrix} x_1\\ x_2\\ .\\ .\\ .\\ x_n\\ \end{bmatrix} = \begin{bmatrix} y_1\\ y_2\\ .\\ .\\ .\\ y_n\\ \end{bmatrix}

Where m > n which means the number of rows is greater than the number of columns or the number of rows is greater than the number of variables.

Solution to the above problem:
There are multiple ways to solve the above problem. One solution involves Moore – Penrose Pseudoinverse. We write the Moore – Penrose pseudoinverse as A^[+}.
We have A^{+}A = I but AA^{+} \neq I unless A has the usual inverse.
So, to solve the problem we proceed as follow:
\overrightarrow{y} = A\overrightarrow{x}


A^{+}\overrightarrow{y} \approx A^{+}A\overrightarrow{x}

A^{+}\overrightarrow{y} \approx I\overrightarrow{x}

A^{+}\overrightarrow{y} \approx \overrightarrow{x}

This is how simply we solve the linear equations using the Moore – Penrose pseudoinverse. The derivation for Moore – Penrose pseudoinverse is beyond the scope of this article. You can go through this link in case you want to know more about it. Here, it is simply presented the method for computing it. The Moore – Penrose pseudoinverse is computed as
A^{+} = (A^{T} A)^{-1} A^{T}

Example:

Consider below 3 linear equations:
x_1 + 3x_2 = 17

5x_1 + 7x_2 = 19

11x_1 + 13x_2 = 23

Equivalently we can write above equations in matrix form as shown below:
A\overrightarrow{x} = \overrightarrow{y}


  \begin{bmatrix}  1 & 3 \\ 5 & 7 \\ 11 & 13 \\ \end{bmatrix} % \begin{bmatrix} x_1\\ x_2\\ \end{bmatrix} = \begin{bmatrix} 17\\ 19\\ 23\\ \end{bmatrix}  

After we compute the Moore - Penrose pseudoinverse using A^{+} = (A^{T} A)^{-1} A^{T}, we will get:

A^{+} = \begin{bmatrix}  -0.5197&-0.2171&0.2368 \\ 0.4276&0.2039&-0.1316 \\ \end{bmatrix} 

Check the matrix product A^{+}A you will get identity matrix I.
A^{+}A = I 

We have,
A\overrightarrow{x} = \overrightarrow{y} 

Then,
A^{+}A\overrightarrow{x} \approx A^{+}\overrightarrow{y} 

\overrightarrow{x} \approx A^{+}\overrightarrow{y} 

 \begin{bmatrix}  x_1\\ x_2\\ \end{bmatrix} \approx \begin{bmatrix}  -0.5197&-0.2171&0.2368 \\ 0.4276&0.2039&-0.1316 \\ \end{bmatrix} % \begin{bmatrix}  7\\ 11\\ \end{bmatrix} 
After you compute above computation, you will get the final answer as
 \begin{bmatrix}  x_1\\ x_2\\ \end{bmatrix} \approx \begin{bmatrix}  -7.51\\ 8.12\\ \end{bmatrix} 

Note: Moore – Penrose pseudoinverse solves the problem in least squared error sense. In general, there is no exact solution to overdetermined problems. So if you cross check the solution you will not get the exact required y but an approx value of y.



My Personal Notes arrow_drop_up

Check out this Author's contributed articles.

If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.

Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.


Article Tags :
Practice Tags :


2


Please write to us at contribute@geeksforgeeks.org to report any issue with the above content.