In this article, we will learn how to install Scrapy in Python on MacOS.
Scrapy is a fast high-level web crawling and web scraping framework used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.
Installation:
Method 1: Using pip to install Scrapy
Follow the below steps to install the SCrapy package on macOS using pip:
Step 1: Install the latest Python3 in MacOS
Step 2: Check if pip3 and python3 are correctly installed.
python3 --version pip3 --version
Step 3: Upgrade your pip to avoid errors during installation.
pip3 install --upgrade pip
Step 4: Enter the following command to install Scrapy using pip3.
pip3 install Scrapy
Method 2: Using setup.py to install Scrapy
Follow the below steps to install the Scrapy package on macOS using the setup.py file:
Step 1: Download the latest source package of Scrapy for python3 from here.
curl https://files.pythonhosted.org/packages/e9/dc/b2f41d802f548fba35c2d7c443ab0ab3df358eebd261f4d30d19ad39658a/Scrapy-2.5.0.tar.gz > scrapy.tar.gz
Step 2: Extract the downloaded package using the following command.
tar -xzvf scrapy.tar.gz
Step 3: Go inside the folder and Enter the following command to install the package.
Note: You must have developer tools for XCode MacOS installed in your system
cd Scrapy-2.5.0 python3 setup.py install
Verifying Scrapy installation on macOS:
Make the following import in your python terminal to verify if the installation has been done properly:
import scrapy
If there is any error while importing the module then is not installed properly.