Multi Dimensional Inputs in Pytorch Linear Method in Python

• Difficulty Level : Expert
• Last Updated : 30 Jan, 2023

In PyTorch, the torch.nn.Linear class is a linear layer that applies a linear transformation to the input data. It is called linear transformation because it applies the linear equation. i.e

Here

• x : input data of one or more dimensions
• A : weight
• b : bias

syntax: torch.nn.Linear(in_features, out_features, bias=True, device=None, dtype=None)

Here:

• in_features : Input features will be an integer value. It denotes the number of variables or columns for the rectangular data frame.
•  out_features : Output features will also be an integer value. It denotes the number of output value. We can get1 or more output values.
• bias : It take the boolean value True or False, For default it is True.
• device (Optional) :  For default it is None. Here we can change of GPU.
• dtype (Optional) :  For default it is None. We can change to any float value.

The torch.nn.Linear class expects the input data to be in the form of a 1-D,2-D, or 3-D tensor, with the last dimension being the input size and the other dimensions representing the batch size and other dimensions.

When working with multi-dimensional input data, the torch.nn.Linear class will apply the linear transformation to the last dimension of the input tensor. It means that the last dimension of the input tensor is considered as input size

In this way, we can use the torch.nn.Linear class to apply a linear transformation to multi-dimensional input data like images, videos, etc. This can be useful in various applications like image classification, object detection, etc.

For example, if you’re working with a 3-D tensor representing a set of images, each image having a number of channels, the last dimension of the tensor will be the number of pixels, and the other dimensions will represent the batch size, and the number of channels. The torch.nn.Linear layer will apply a linear transformation to the number of pixels, and the other dimensions will be preserved.

Example 1:

Step 1: Import the necessary packages

Python3

 `# Import the library``import` `torch``import` `torch.nn as nn`

Step 2: Define the Input Tensor

Python3

 `# Define the input data as a 2-D tensor``x ``=` `torch.tensor([[``0.``,``1.``],[``2.``,``3.``],[``4.``,``5.``]])``#Shape``print``(``'input_data:'``, x.shape)`

Output:

`input_data: torch.Size([3, 2])`

Step 3:  Define the output size and Create the Linear transformation layer

Python3

 `# Create an instance of the nn.Linear class``output_size ``=` `1``linear_layer ``=` `nn.Linear(in_features``=``2``,``                         ``out_features``=``output_size,``                         ``bias ``=` `True``,``                         ``dtype``=` `torch.``float``)`

Step 4: Apply the linear transformation to the input data. and print the output.

Python3

 `# Apply the linear transformation to the input data``y ``=` `linear_layer(x)`` ` `#print the outputs``print``(``'output:'``,y)`` ` `#print the outputs shape``print``(``'output Shape:'``, y.shape)`

Output:

```output:
tensor([[-0.5798],
[-2.1357],
output Shape: torch.Size([3, 1])```

Python3

 `# Linear layer weight``A ``=` `linear_layer.weight``#Bias``b ``=` `linear_layer.bias`` ` `#Output``y ``=` `x @ A.transpose(``0``,``1``) ``+` `b`` ` `#print the outputs``print``(y)`

Output:

```tensor([[-0.5798],
[-2.1357],

Python3

 `# Import the library``import` `torch``import` `torch.nn as nn`` ` `# Define the input data as a 3-D tensor``x ``=` `torch.tensor([[[``0.``,``1.``,``2.``],``                   ``[``3.``,``4.``,``5.``],``                   ``[``6.``,``7.``,``8.``]]])`` ` `print``(``'input_data:'``, x.shape)`` ` `# Create an instance of the nn.Linear class``output_size ``=` `5``linear_layer ``=` `nn.Linear(in_features``=``x.size()[``-``1``],``                         ``out_features``=``output_size,``                         ``bias ``=` `True``,``                         ``dtype``=` `torch.``float``)`` ` `# Apply the linear transformation to the input data``y ``=` `linear_layer(x)``#print the outputs``print``(``'\noutput:\n'``,y)`` ` `#print the outputs shape``print``(``'\noutput Shape:'``, y.shape)`

Output:

```input_data: torch.Size([1, 3, 3])

output:
tensor([[[ 0.8020,  0.7344, -0.1366,  0.7514, -0.0804],
[ 3.7831,  1.5299, -0.1147, -0.1602,  0.8117],
[ 6.7643,  2.3254, -0.0927, -1.0718,  1.7038]]],

output Shape: torch.Size([1, 3, 5])```

Check the Results:

Python3

 `# Linear layer weight``A ``=` `linear_layer.weight``#Bias``b ``=` `linear_layer.bias`` ` `#Output y= xA^T + b``##Matrix product of two tensors``y ``=` `x.matmul(A.transpose(``0``,``1``)) ``+` `b`` ` `#print the outputs``print``(y)`

Output:

```tensor([[[ 0.8020,  0.7344, -0.1366,  0.7514, -0.0804],
[ 3.7831,  1.5299, -0.1147, -0.1602,  0.8117],
[ 6.7643,  2.3254, -0.0927, -1.0718,  1.7038]]],

Python3

 `# Import the library``import` `torch``import` `torch.nn as nn`` ` `# Define the input data as a 3-D tensor``batch_size ``=` `2``num_channels ``=` `3``input_size ``=` `4`` ` `input_data ``=` `torch.randn(size ``=``(batch_size, num_channels, input_size))``print``(``'input Shape :'``, input_data.shape)``print``(``'Input\n'``,input_data)`` ` `# Create an instance of the nn.Linear class``output_size ``=` `2``linear_layer ``=` `nn.Linear(in_features``=``input_data.size()[``-``1``],``                         ``out_features``=``output_size,``                         ``bias ``=` `True``)`` ` `# Apply the linear transformation to the input data``output ``=` `linear_layer(input_data)``print``(``'\noutput shape:'``, output.shape)``print``(``'output:\n'``,output)`

Output

```input Shape : torch.Size([2, 3, 4])
Input
tensor([[[ 0.2777, -0.1791, -0.0452,  1.4240],
[-0.3565, -1.0471,  0.2848,  0.4885],
[ 0.4119,  0.4591, -0.9893,  0.7301]],

[[-0.8063, -0.7165,  1.6583,  1.2563],
[ 0.0972, -0.3562,  1.1938,  0.4868],
[ 2.1877,  0.3338,  0.7358, -1.3842]]])

output shape: torch.Size([2, 3, 2])
output:
tensor([[[ 0.5222,  0.1193],
[ 0.0817, -0.4504],
[ 0.2597,  0.4163]],

[[ 0.7127, -0.3072],
[ 0.4723, -0.3006],
```

Check the Results:

Python3

 `# Linear layer weight``A ``=` `linear_layer.weight``#Bias``b ``=` `linear_layer.bias`` ` `#Output y= xA^T + b``##Matrix product of two tensors``y ``=` `input_data.matmul(A.transpose(``0``,``1``)) ``+` `b`` ` `#print the outputs``print``(y)`

Output:

```tensor([[[ 0.5222,  0.1193],
[ 0.0817, -0.4504],
[ 0.2597,  0.4163]],

[[ 0.7127, -0.3072],
[ 0.4723, -0.3006],

Explanation:

Here, input_data is 3D tensor having batch_size, num_channels and input_size as dimensions. torch.nn.Linear(in_features=input_data.size()[-1], out_features=output_size) creates a Linear layer with input size being the last dimension of input_data (i.e. input_size) and output size being output_size.

The output of this operation will be a 3D tensor having dimensions batch_size, num_channels and output_size. The output size is defined by the out_features parameter when creating an instance of the nn.Linear class.

It’s also important to note that this script is only for linear layers and for model building, we need to perform forward and backward pass, And there a non-linear activation function like ReLU, tanh, etc. will be added based on the use case, which to decide the weights and biases of the model.

You can also change the shape of the input_data as per your requirement, for example, you can use a 4D tensor for 3D convolution, and so on.

My Personal Notes arrow_drop_up