In PyTorch, the torch.nn.Linear class is a linear layer that applies a linear transformation to the input data. It is called linear transformation because it applies the linear equation. i.e
Here
- x : input data of one or more dimensions
- A : weight
- b : bias
syntax: torch.nn.Linear(in_features, out_features, bias=True, device=None, dtype=None)
Here:
- in_features : Input features will be an integer value. It denotes the number of variables or columns for the rectangular data frame.
- out_features : Output features will also be an integer value. It denotes the number of output value. We can get1 or more output values.
- bias : It take the boolean value True or False, For default it is True.
- device (Optional) : For default it is None. Here we can change of GPU.
- dtype (Optional) : For default it is None. We can change to any float value.
The torch.nn.Linear class expects the input data to be in the form of a 1-D,2-D, or 3-D tensor, with the last dimension being the input size and the other dimensions representing the batch size and other dimensions.
When working with multi-dimensional input data, the torch.nn.Linear class will apply the linear transformation to the last dimension of the input tensor. It means that the last dimension of the input tensor is considered as input size
In this way, we can use the torch.nn.Linear class to apply a linear transformation to multi-dimensional input data like images, videos, etc. This can be useful in various applications like image classification, object detection, etc.
For example, if you’re working with a 3-D tensor representing a set of images, each image having a number of channels, the last dimension of the tensor will be the number of pixels, and the other dimensions will represent the batch size, and the number of channels. The torch.nn.Linear layer will apply a linear transformation to the number of pixels, and the other dimensions will be preserved.
Example 1:
Step 1: Import the necessary packages
Python3
import torch
import torch.nn as nn
|
Step 2: Define the Input Tensor
Python3
x = torch.tensor([[ 0. , 1. ],[ 2. , 3. ],[ 4. , 5. ]])
print ( 'input_data:' , x.shape)
|
Output:
input_data: torch.Size([3, 2])
Step 3: Define the output size and Create the Linear transformation layer
Python3
output_size = 1
linear_layer = nn.Linear(in_features = 2 ,
out_features = output_size,
bias = True ,
dtype = torch. float )
|
Step 4: Apply the linear transformation to the input data. and print the output.
Python3
y = linear_layer(x)
print ( 'output:' ,y)
print ( 'output Shape:' , y.shape)
|
Output:
output:
tensor([[-0.5798],
[-2.1357],
[-3.6916]], grad_fn=<AddmmBackward0>)
output Shape: torch.Size([3, 1])
Check the Result:
Python3
A = linear_layer.weight
b = linear_layer.bias
y = x @ A.transpose( 0 , 1 ) + b
print (y)
|
Output:
tensor([[-0.5798],
[-2.1357],
[-3.6916]], grad_fn=<AddBackward0>)
Example 2
Python3
import torch
import torch.nn as nn
x = torch.tensor([[[ 0. , 1. , 2. ],
[ 3. , 4. , 5. ],
[ 6. , 7. , 8. ]]])
print ( 'input_data:' , x.shape)
output_size = 5
linear_layer = nn.Linear(in_features = x.size()[ - 1 ],
out_features = output_size,
bias = True ,
dtype = torch. float )
y = linear_layer(x)
print ( '\noutput:\n' ,y)
print ( '\noutput Shape:' , y.shape)
|
Output:
input_data: torch.Size([1, 3, 3])
output:
tensor([[[ 0.8020, 0.7344, -0.1366, 0.7514, -0.0804],
[ 3.7831, 1.5299, -0.1147, -0.1602, 0.8117],
[ 6.7643, 2.3254, -0.0927, -1.0718, 1.7038]]],
grad_fn=<ViewBackward0>)
output Shape: torch.Size([1, 3, 5])
Check the Results:
Python3
A = linear_layer.weight
b = linear_layer.bias
y = x.matmul(A.transpose( 0 , 1 )) + b
print (y)
|
Output:
tensor([[[ 0.8020, 0.7344, -0.1366, 0.7514, -0.0804],
[ 3.7831, 1.5299, -0.1147, -0.1602, 0.8117],
[ 6.7643, 2.3254, -0.0927, -1.0718, 1.7038]]],
grad_fn=<AddBackward0>)
Example 3:
Python3
import torch
import torch.nn as nn
batch_size = 2
num_channels = 3
input_size = 4
input_data = torch.randn(size = (batch_size, num_channels, input_size))
print ( 'input Shape :' , input_data.shape)
print ( 'Input\n' ,input_data)
output_size = 2
linear_layer = nn.Linear(in_features = input_data.size()[ - 1 ],
out_features = output_size,
bias = True )
output = linear_layer(input_data)
print ( '\noutput shape:' , output.shape)
print ( 'output:\n' ,output)
|
Output
input Shape : torch.Size([2, 3, 4])
Input
tensor([[[ 0.2777, -0.1791, -0.0452, 1.4240],
[-0.3565, -1.0471, 0.2848, 0.4885],
[ 0.4119, 0.4591, -0.9893, 0.7301]],
[[-0.8063, -0.7165, 1.6583, 1.2563],
[ 0.0972, -0.3562, 1.1938, 0.4868],
[ 2.1877, 0.3338, 0.7358, -1.3842]]])
output shape: torch.Size([2, 3, 2])
output:
tensor([[[ 0.5222, 0.1193],
[ 0.0817, -0.4504],
[ 0.2597, 0.4163]],
[[ 0.7127, -0.3072],
[ 0.4723, -0.3006],
[-0.0074, -0.4597]]], grad_fn=<ViewBackward0>)
Check the Results:
Python3
A = linear_layer.weight
b = linear_layer.bias
y = input_data.matmul(A.transpose( 0 , 1 )) + b
print (y)
|
Output:
tensor([[[ 0.5222, 0.1193],
[ 0.0817, -0.4504],
[ 0.2597, 0.4163]],
[[ 0.7127, -0.3072],
[ 0.4723, -0.3006],
[-0.0074, -0.4597]]], grad_fn=<AddBackward0>)
Explanation:
Here, input_data is 3D tensor having batch_size, num_channels and input_size as dimensions. torch.nn.Linear(in_features=input_data.size()[-1], out_features=output_size) creates a Linear layer with input size being the last dimension of input_data (i.e. input_size) and output size being output_size.
The output of this operation will be a 3D tensor having dimensions batch_size, num_channels and output_size. The output size is defined by the out_features parameter when creating an instance of the nn.Linear class.
It’s also important to note that this script is only for linear layers and for model building, we need to perform forward and backward pass, And there a non-linear activation function like ReLU, tanh, etc. will be added based on the use case, which to decide the weights and biases of the model.
You can also change the shape of the input_data as per your requirement, for example, you can use a 4D tensor for 3D convolution, and so on.
Last Updated :
30 Jan, 2023
Like Article
Save Article
Share your thoughts in the comments
Please Login to comment...