Amit Kumar Malaker
Posted on July 19, 2021
Pre-context:
The thing is I love fast things. And I am sure the reader also pursue same nature like me. Hence, pypy3 is the choice.
Fedora is nice but for me it is new and new thing is not always hassle free so to make your life better this instructions will be life saving.
First install pypy3 and respective pip pkgmanager:
sudo dnf install pypy3
wget https://bootstrap.pypa.io/get-pip.py -o get-pip.py
pypy3 get-pip.py
Then install the below dependencies which are the root cause of scrapy installation failures. So do it carefully.
sudo dnf install pypy3-devel
sudo dnf install libxml2-devel
sudo dnf install libxslt-devel
sudo dnf install @development-tools
sudo dnf install libffi-devel
install them if you find them: [zlib1g-dev, libssl-dev] (optional)
Now create virtual environment:
pypy3 -m venv venv
Activate the virtual environment:
source venv/bin/activate
Now install scrapy:
pypy3 -m pip install setuptools
pypy3 -m pip install scrapy
Hopefully scrapy will be installed successfully.
Now create scrapy project.
scrapy startproject demo
Create a spider:
cd demo
scrapy genspider demo demo
In fine, this is the way to have our beloved scrapy working in fedora.
If you have any problem you can comment below. I will try my best to resolve it.
Posted on July 19, 2021
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.