Python PyTorch – linalg.norm() method

PyTorch linalg.norm() method computes a vector or matrix norm. Norm is always a non-negative real number which is a measure of the magnitude of the matrix. It accepts a vector or matrix or batch of matrices as the input. It supports inputs of only float, double, cfloat, and cdouble dtypes. We will be using the following syntax to compute the vector or matrix norm.

Syntax: torch.linalg.norm(A, ord=None, dim=None, keepdim=False, *, out=None, dtype=None)

Parameters:

• A:  the input tensor with the shape (*, n) or (*, m, n) where * is zero or more batch dimensions.
• ord: order of norm.
• dim: it is the dimensions over which to compute the norm. It is optional and may be an integer value or tuple of integer values.
• keepdim:  optional boolean value. If it is True, the reduced dimensions are retained in the result as dimensions with size one.
• out: optional output.
• dtype: an optional keyword argument. If it is specified, the input is cast to dtype and then norm is computed, and the type returned tensor will be dtype.

Return: it returns real-valued tensor, even when A is complex.

This method computes a vector norm if dim is an int and matrix norm if dim is a 2-tuple. If both dim and ord are None, the input tensor A will be flattened to 1D vector and the 2-norm will be computed. If ord != None and dim= None , A must be 1D or 2D.

Example 1:

In the example below we find the vector norm using the torch.linalg.norm() method. We define a 1D tensor (vector) and compute the norm with the default parameters of the method. We also reshape the 1D tensor to 2D and compute the norm. Here, the norm of 1D Tensor and 2D Tensor is the same. The elements of both the tensors are the same and the default parameters are ord = None, dim =None. So the 2D tensor is flattened to 1D and the vector norm is computed. Notice that the norm is the non-negative real-valued number.

Python3

# Python program to compute vector norm
# importing libraries
import torch
import torch.linalg as la

# define input a 1D tensor
a = torch.tensor([-3., -4., 1., 0., 3., 2., -1., -7.])
print("1D Tensor:\n", a)

# computing the norm
norm = la.norm(a)

# print the computed norm
print("Norm:", norm)

# reshape the tensor to 2D tensor
a = a.reshape(2, 4)
print("2D Tensor:\n", a)

# again compute the norm
norm = la.norm(a)

# print the norm
print("Norm:", norm)

Output:

1D Tensor:
 tensor([-3., -4.,  1.,  0.,  3.,  2., -1., -7.])
Norm: tensor(9.4340)
2D Tensor:
 tensor([[-3., -4.,  1.,  0.],
        [ 3.,  2., -1., -7.]])
Norm: tensor(9.4340)

Example 2:

In the example below we find the matrix norm using the torch.linalg.norm() method. We define a 2D tensor (matrix) and compute the matrix norm with different ord values. And the input tensor is a 2D tensor (matrix). The computed matrix norms are different for different ord values. Please note that ord = 0  is not supported for matrix norm and  ord =”fro” , ord =  “nuc” are not supported for vector (1D tensor) norm. Also, note that the norm is a non-negative real-valued number.

Python3

# Python program to compute norm
# importing libraries
import torch
import torch.linalg as la

# define input a 2D tensor (matrix)
a = torch.tensor([[1., 2., -3.],
                  [-4., 5., 6.],
                  [9., -7., 8.]])
print("Tensor:\n", a)

# compute and print the computed norm
print("\nNorm with different ord:\n")
print("Norm with ord= None:",
      la.norm(a, ord=None))
print("Norm ord= 'fro':",
      la.norm(a, ord='fro'))
print("Norm with ord= float('inf'):",
      la.norm(a, ord=float('inf')))
print("Norm with ord= -float('inf'):",
      la.norm(a, ord=-float('inf')))
print("Norm with ord= 'nuc'):",
      la.norm(a, ord='nuc'))

# print("Norm with ord= 0:", la.norm(a, ord= 0))
# ord=0 not supported for matrix norm
print("Norm with ord= 1:",
      la.norm(a, ord=1))
print("Norm with ord= -1:",
      la.norm(a, ord=-1))
print("Norm with ord= 2:",
      la.norm(a, ord=2))
print("Norm with ord= -2:",
      la.norm(a, ord=-1))

Output:

Tensor:
 tensor([[ 1.,  2., -3.],
        [-4.,  5.,  6.],
        [ 9., -7.,  8.]])

Norm with different ord:

Norm with ord= None: tensor(16.8819)
Norm ord= 'fro': tensor(16.8819)
Norm with ord= float('inf'): tensor(24.)
Norm with ord= -float('inf'): tensor(6.)
Norm with ord= 'nuc'): tensor(25.4331)
Norm with ord= 1: tensor(17.)
Norm with ord= -1: tensor(14.)
Norm with ord= 2: tensor(14.2151)
Norm with ord= -2: tensor(14.)

Example 3:

In the example below we find the norms using the torch.linalg.norm() method. We define a 2D tensor (matrix) and compute the norm with different dim and ord values. And the input tensor is a 2D tensor (matrix). The computed matrix norms are different for different ord and dim values. Please note that the norm with dim = 0 is computed along with the columns and the norm with dim=1 is computed along the rows. Also, note that norm is a non-negative real-valued number.

Python3

# Python program to compute matrix norm
# importing libraries
import torch
import torch.linalg as la

# define input a 2D tensor (matrix)
a = torch.tensor([[1., 2., -3.],
                  [-4., 5., 6.]])
print("Tensor:\n", a)

# compute and print the computed norm
# with different ord and dim
print("Norm with dim=0:",
      la.norm(a, dim=0))
print("Norm with dim=1:",
      la.norm(a, dim=1))
print("Norm with ord=1, dim=1:",
      la.norm(a, ord=1, dim=1))
print("Norm with ord=-1, dim=1:",
      la.norm(a, ord=-1, dim=1))
print("Norm with ord=1, dim=0:",
      la.norm(a, ord=1, dim=0))
print("Norm with ord=-1, dim=0:",
      la.norm(a, ord=1, dim=1))
print("Norm with dim=(0,1):",
      la.norm(a, dim=(0, 1)))

Output:

Tensor:
 tensor([[ 1.,  2., -3.],
        [-4.,  5.,  6.]])
Norm with dim=0: tensor([4.1231, 5.3852, 6.7082])
Norm with dim=1: tensor([3.7417, 8.7750])
Norm with ord=1, dim=1: tensor([ 6., 15.])
Norm with ord=-1, dim=1: tensor([0.5455, 1.6216])
Norm with ord=1, dim=0: tensor([5., 7., 9.])
Norm with ord=-1, dim=0: tensor([ 6., 15.])
Norm with dim=(0,1): tensor(9.5394)