Python PyTorch linalg.svd() method

PyTorch linalg.svd() method computes the singular value decomposition (SVD) of a matrix. 2D tensors are matrices in PyTorch. This method supports both real and complex-valued matrices (float, double, cfloat, and cdouble dtypes).  It takes input a matrix or a batch of matrices and returns decomposition as a named tuple (U, S, VT). I have listed some properties of U, S, and VT as below-

• If input matrix A is a real-valued matrix, both U and VT are orthogonal.
• If input matrix A is a complex-valued matrix, U and VT are unitary.
• S is always real-valued for both real or complex-valued input matrix A. The singular values (elements of S) are arranged in descending order.

Following is the syntax-

Syntax: torch.linalg.svd(A, full_matrices=True, *, out=None)

Parameters:

• A: the input matrix (tensor or batch of tensors).
• full_matrices: an optional boolean value. If True full SVD is computed. If False, reduced SVD is computed.
• out: the output tuple of three tensors.

Return: It returns three tensors as a named tuple (U, S, VT).

Let’s understand the torch.linalg.svd() method with the help of some Python program examples.

Example 1:

In this example, we compute the full singular value decomposition of a real-valued matrix using torch.linalg.svd(). Here, the U and VT matrices are square matrices, and the size of S is min(2,3). Matrices U and VT are orthogonal. You can check the orthogonality of these matrices using U@U.t() and VT@VT.t().

Python3

# Python program to demonstrate torch.linalg.svd()
# method for a real-valued matrix
# importing torch
import torch

# creating a real-valued matrix (2-D tensor)
Mat = torch.tensor([[1.,2.,3.],
                    [3.,-2,-7]])
# printing the matrix
print("Matrix:\n", Mat)

# computing SVD of the matrix Mat
U, S, VT = torch.linalg.svd(Mat)

# printing U, S, and Vh
print("U:\n",U)
print("Singular Values (S):\n",S)
print("VT:\n",VT)
print("Shape of U, S and VT:\n", U.shape, S.shape, VT.shape)

Output:

Matrix:
 tensor([[ 1.,  2.,  3.],
        [ 3., -2., -7.]])
U:
 tensor([[-0.3625, -0.9320],
        [ 0.9320, -0.3625]])
Singular Values (S):
 tensor([8.3999, 2.3329])
VT:
 tensor([[ 0.2897, -0.3082, -0.9061],
        [-0.8657, -0.4882, -0.1107],
        [ 0.4082, -0.8165,  0.4082]])
Shape of U, S and VT:
 torch.Size([2, 2]) torch.Size([2]) torch.Size([3, 3])

Example 2:

In the example below we compute the reduced singular value decomposition of a real-valued matrix using torch.linalg.svd(). Here, the reduced SVD is computed. Notice that U is square and VT is not, and the size of S is min(3,4). As the input matrix is real-valued matrices U and VT should be orthogonal. You can check the orthogonality of these matrices using U@U.t() and VT@VT.t(). What if the input matrix Mat in the above code is a 4×3 matrix? Yes!!!. VT is square and U is not.

Python3

# Python program to demonstrate torch.linalg.svd()
# method for a real-valued matrix with
# full_matrices=False
# importing torch
import torch

# creating a real-valued matrix (2-D tensor)
Mat = torch.rand(3, 4)
# printing the matrix
print("Matrix:\n", Mat)

# computing SVD of the matrix Mat
U, S, VT = torch.linalg.svd(Mat, full_matrices=False)

# printing U, S, and Vh
print("U:\n", U)
print("Singular Values (S):\n", S)
print("VT:\n", VT)
print("Shape of U, S and VT:\n", U.shape, S.shape, VT.shape)

Output:

Please note that as we are creating an input matrix using a random number generator, you may get a different matrix from the output-

Matrix:
 tensor([[0.5280, 0.3108, 0.1215, 0.8151],
        [0.9640, 0.4199, 0.3913, 0.2239],
        [0.7886, 0.0702, 0.7123, 0.6120]])
U:
 tensor([[-0.4984, -0.7911, -0.3546],
        [-0.5810,  0.6083, -0.5407],
        [-0.6435,  0.0634,  0.7628]])
Singular Values (S):
 tensor([1.8411, 0.5512, 0.4225])
VT:
 tensor([[-0.7227, -0.2412, -0.4053, -0.5052],
        [ 0.3969,  0.0254,  0.3395, -0.8524],
        [-0.2531, -0.6715,  0.6834,  0.1343]])
Shape of U, S and VT:
 torch.Size([3, 3]) torch.Size([3]) torch.Size([3, 4])

Example 3:

In the example below we compute the full SVD of a complex-valued matrix. Notice that the S is real-valued and U and VT are complex-valued.

Python3

# Python program to demonstrate torch.linalg.svd()
# method for a complex-valued matrix
# importing torch
import torch

# creating a complex-valued matrix (2-D tensor)
Mat = torch.randn(3, 2, dtype=torch.cfloat)
# printing the matrix
print("Matrix:\n", Mat)

# computing SVD of the matrix Mat
U, S, VT = torch.linalg.svd(Mat)

# printing U, S, and Vh
print("U:\n", U)
print("Singular Values (S):\n", S)
print("VT:\n", VT)
print("Shape of U, S and VT:\n", U.shape, S.shape, VT.shape)

Output:

Please note that as we are creating an input matrix using a random number generator, you may get a different matrix from the output below

Matrix:
 tensor([[-0.4095-0.8878j,  0.3983+0.5446j],
        [-1.3408+1.1268j,  0.3193+0.9775j],
        [ 0.8876+0.6970j, -0.9217-0.5416j]])
U:
 tensor([[-0.2687-0.3263j, -0.3146+0.2721j,  0.7890+0.1605j],
        [-0.6525+0.3324j, -0.1834-0.6499j,  0.0518+0.0713j],
        [ 0.5008+0.1853j,  0.3777-0.4778j,  0.5800-0.0863j]])
Singular Values (S):
 tensor([2.4929, 1.3183])
VT:
 tensor([[ 0.8916+0.0000j, -0.2928-0.3453j],
        [-0.4527+0.0000j, -0.5767-0.6800j]])
Shape of U, S and VT:
 torch.Size([3, 3]) torch.Size([2]) torch.Size([2, 2])