PyTorch Tensor vs NumPy Array

PyTorch and NumPy can help you create and manipulate multidimensional arrays. This article covers a detailed explanation of how the tensors differ from the NumPy arrays.

What is a PyTorch Tensor?

PyTorch tensors are the data structures that allow us to handle multi-dimensional arrays and perform mathematical operations on them. In other words, a PyTorch tensor is a multi-dimensional array that can hold data of a uniform data type. It is similar to NumPy arrays. These have different ranks that represent the scalars (0D), vectors (1D), matrices (2D), or higher-dimensional arrays (nD). They have various data types like floating-point numbers (float32, float64), integers (int32, int64), and others, which makes them flexible. Thus, tensors act as the backbone of the PyTorch model.

What is a NumPy array?

A NumPy array is a fundamental data structure in the NumPy library for Python, representing multi-dimensional arrays of homogeneous data. It provides efficient storage and operations on large datasets, enabling numerical computations such as linear algebra, statistical analysis, and data manipulation.

You might think that both the PyTorch Tensors and NumPy Arrays are almost the same in terms of functionality. So let us move on to the differences between them.

Tensors and NumPy Array: Key-Differences

Feature	PyTorch Tensors	NumPy Arrays
Definition	Tensors are multi-dimensional arrays with uniform data types with more optimization for Deep Learning	They are also multi-dimensional arrays with a uniform data type with less support for Deep Learning.
Syntax and Interface	You can use the torch.tensor() method to create the Tensors.	To create the NumPy Arrays, the np.array() method is used.
Automatic Differentiation	It supports the Built-in automatic differentiation using PyTorch’s Autograd module.	There is no support for automatic differentiation.
GPU Support	We can integrate it with CUDA-enabled GPUs for accelerated computation.	It provides Limited support for GPU. Thus, we need additional libraries for GPU.
Dynamic Computational Graph	It supports dynamic computation graphs in which the graph can be changed at the run time.	It supports the Static computation graph in which the computation graph is defined before execution.
Performance	It supports efficient GPU acceleration for deep learning tasks.	It is efficient for general-purpose numerical computations but less optimized for deep learning.
Deployment	It supports the deployment learning models in production environments.	We require additional steps for deployment and integration with deep learning frameworks.
Memory Management	It has Automatic memory management with garbage collection.	It has Manual memory management. Thus, we need to implement explicit memory deallocation.
Integration with Deep Learning Frameworks	It supports Native integration with PyTorch’s deep learning ecosystem for seamless model development.	It also requires additional steps for integration with deep learning frameworks like TensorFlow or Keras.
Parallelization	It supports parallel operations across multiple CPU or GPU cores.	The Parallel operations depend on the underlying linear algebra libraries like BLAS and CPU/GPU hardware.

Creating and Element wise operations in Pytorch Tensors and Numpy Arrays

1. Pytorch Tensors

In the below code snippet, we are importing the torch module to create the Tensors and then performing the element-wise operations on them as illustrated below:

Python

import torch
# Create a PyTorch tensor from a Python list
tensor_list = torch.tensor([1, 2, 3, 4, 5])
print("PyTorch Tensor from List:")
print(tensor_list)
# Create a PyTorch tensor of zeros with a specified shape
tensor_zeros = torch.zeros(2, 3)
print("\nPyTorch Tensor of Zeros:")
print(tensor_zeros)
# Perform element-wise operations on PyTorch tensors
tensor_a = torch.tensor([1, 2, 3])
tensor_b = torch.tensor([4, 5, 6])
tensor_sum = tensor_a + tensor_b
print("\nElement-wise Sum of Tensors:")
print(tensor_sum)

Output:

PyTorch Tensor from List:
tensor([1, 2, 3, 4, 5])
PyTorch Tensor of Zeros:
tensor([[0., 0., 0.],
        [0., 0., 0.]])
Element-wise Sum of Tensors:
tensor([5, 7, 9])

2. NumPy Array

Now, let us create the NumPy Array and perform the operations. First, we have to import the NumPy module and then create the arrays. Then, we have to perform the element-wise operations. This is illustrated in the below code snippet.

Python

import numpy as np
# Create a NumPy array from a Python list
array_list = np.array([1, 2, 3, 4, 5])
print("NumPy Array from List:")
print(array_list)
# Create a NumPy array of zeros with a specified shape
array_zeros = np.zeros((2, 3))
print("\nNumPy Array of Zeros:")
print(array_zeros)
# Perform element-wise operations on NumPy arrays
array_a = np.array([1, 2, 3])
array_b = np.array([4, 5, 6])
array_sum = array_a + array_b
print("\nElement-wise Sum of Arrays:")
print(array_sum)

Output:

NumPy Array from List:
[1 2 3 4 5]
NumPy Array of Zeros:
[[0. 0. 0.]
 [0. 0. 0.]]
Element-wise Sum of Arrays:
[5 7 9]

Implementing Functions in NumPy Array and Tensors

• rand() function: This function generates arrays or tensors filled with random values sampled from a uniform distribution over a specified interval, typically [0, 1). The rand function is part of the random module and takes one or more arguments representing the dimensions of the output array. Its implementation using Tensors is shown in the below code snippet.

Python

import torch
import numpy as np

# Generate a 1D tensor of size 5 filled with random numbers
random_tensor = torch.rand(5)
print('PyTorch Tensor')
print(random_tensor)

# Generate a 1D array of 5 random numbers
random_array = np.random.rand(5)
print('Numpy Array')
print(random_array)

Output:

PyTorch Tensor
tensor([0.7202, 0.5758, 0.8367, 0.7984, 0.7678])
Numpy Array
[0.34504422 0.54502723 0.60215318 0.60033514 0.34551743]

• seed() function: This function initializes the random number generator with a specified seed value. When you set the seed, you can ensure that the same sequence of random numbers will be generated every time the code is run with the seed.

Python

import torch
import numpy as np

# Set the seed to 42
torch.manual_seed(42)
# Generate a random tensor
random_tensor = torch.rand(5)
print("Random Pytorch Tensor:",random_tensor)

# Set the seed to 42
np.random.seed(42)
# Generate a random array
random_array = np.random.rand(5)
print("Random Numpy Array:",random_array)

Output:

Random Pytorch Tensor: tensor([0.8823, 0.9150, 0.3829, 0.9593, 0.3904])
Random Numpy Array: [0.37454012 0.95071431 0.73199394 0.59865848 0.15601864]

• Reshaping function: In Reshaping the array or tensor, the dimensions are changed while preserving the total number of elements. You can use this operation to rearrange the data so that it can be fit for the different computational tasks or algorithms.

For Tensors, you can use the view() method to reshape them as illustrated in the below code.

Python

import torch
import numpy as np

# Create a 1D tensor
tensor = torch.arange(12)
print('Pytorch Tensor:', tensor)
# Reshape the tensor into 3x4 matrix
reshaped_tensor = tensor.view(3, 4)
print("Reshaped Tensor")
print(reshaped_tensor)

# Create 1D array
arr = np.arange(12)
print('Numpy Array:', arr)
# Reshape the array into 3x4 matrix
reshaped_arr = arr.reshape(3, 4)
print("Reshaped Array")
print(reshaped_arr)

Output:

Pytorch Tensor: tensor([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11])
Reshaped Tensor
tensor([[ 0,  1,  2,  3],
        [ 4,  5,  6,  7],
        [ 8,  9, 10, 11]])
Numpy Array: [ 0  1  2  3  4  5  6  7  8  9 10 11]
Reshaped Array
[[ 0  1  2  3]
 [ 4  5  6  7]
 [ 8  9 10 11]]

• Slicing Operation: Slicing means to extract the subset of elements from the original array based on specified indices or ranges. In the below code snippet, we are creating the Tensor and then slicing it by specifying the row and column index inside the square brackets([]).

Python

import torch
import numpy as np

# Create 2D tensor
tensor = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
print('Pytorch Tensor:',tensor)
# Slice the tensor to extract the sub-tensor
sub_tensor = tensor[1:, :2]
print("Tensor after the slicing")
print(sub_tensor)

# Create 2D array
arr = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
print('NumPy Array:',arr)
# Slice the array to extract a subarray
sub_arr = arr[1:, :2]
print("array after slicing")
print(sub_arr)

Output:

Pytorch Tensor: tensor([[1, 2, 3],
        [4, 5, 6],
        [7, 8, 9]])
Tensor after the slicing
tensor([[4, 5],
        [7, 8]])
NumPy Array: [[1 2 3]
 [4 5 6]
 [7 8 9]]
array after slicing
[[4 5]
 [7 8]]

Conclusion

In conclusion, while both PyTorch tensors and NumPy arrays are powerful tools for numerical computation in Python. PyTorch tensors support integration with deep learning frameworks, automatic differentiation, and GPU acceleration. Thus, it is mainly used for Deep Learning Framework.

On other hand, NumPy arrays are widely used in scientific computing and data analysis. It supports various mathematical operations with a wide range of libraries. Therefore, understanding the differences between PyTorch tensors and NumPy arrays helps us to effectively choose right tool as per the requirements.

Frequently Asked Questions

Is NumPy Array faster than PyTorch Tensors?

In general, NumPy arrays and PyTorch tensors offer similar performance for most operations. However, PyTorch tensors may have some advantages in some cases like the deep learning tasks. This is due to optimizations for GPU acceleration and automatic differentiation.

How to install the PyTorch using the pip?

To install the PyTorch, execute command ‘pip install torch torchvision’ It will install the PyTorch and torchvision. PyTorch includes various tools for Model building and torchvision that consist of popular datasets and model architectures.

What is the difference between the PyTorch and TensorFlow?

PyTorch and TensorFlow are both popular deep learning frameworks with Dynamic Graph Computation. However, they both differ in terms of performance metrics like training time and efficiency. For example, training time of TensorFlow is high but memory is optimized. Also, PyTorch uses TorchScript library for tensor manipulation and TensorFlow uses its library for TensorFlow manipulation.

What does loss function mean in PyTorch?

The Loss Function measures how well a model’s predictions match the ground truth labels in a training dataset. To improve the model’s performance, we have to minimize loss function using algorithms like stochastic gradient descent (SGD). Examples of the Loss Function are oss functions in PyTorch including mean squared error (MSE), cross-entropy loss, and negative log-likelihood loss, etc.