Scraping is the process of collection of web metadata or web information through web crawlers. We can get the links associated with the domain, can also retrieve the JavaScript file links, and many more. For performing web scraping we use the Scrapy library. It is purely written in Python. In this article, we will look into the process of installing the Scrapy module on the Linux Operating System.
Requirements:
Installing Python-Scrapy package on Linux using PIP
To install the Python-Scrapy package in Linux we have to follow the following steps:
Step 1: First of all, we will install Python3 on our Linux Machine. Use the following command in the terminal to install the latest version of Python3.
sudo apt-get install python3
Step 2: Now, install the pip module which is required to install and manage the packages in Python3. So use the following command for installation:
sudo apt install python3-pip
Step 3: Now, install the Python-Scrapy package with the help of the following command:
sudo pip3 install scrapy
or
sudo apt-get install python3-scrapy
Verifying Python-Scrapy package Installation on Linux using PIP
To verify if the Python-Scrapy package has been successfully installed in your system run the below command in Terminal:
python3 -m pip show scrapy
You’ll get the below message if the installation process is completed successfully: