TheLazy.Dev
Portfolio GitHub

Blog

python |docker |pip3 |install

Python: Docker image build -- install required packages via requirements.txt vs editable install.

behainguyen

Be Hai Nguyen

Posted on July 22, 2022

Python: Docker image build -- install required packages via requirements.txt vs editable install.

Install via requirements.txt means using this image build step command “RUN pip3 install -r requirements.txt”. Editable install means using the “RUN pip3 install -e .” command. I've experienced that install via requirements.txt resulted in images that do not run, whereas using editable install resulted in images that do work as expected. I'm presenting my findings in this post.

The first Docker tutorial I took was Learn to build and deploy your distributed applications easily to the cloud with Docker, it's an excellent tutorial and I did successfully complete all of it. Then I did this Build your Python image -- only this first part.

What's common between them are -- in my observations:

  1. The Python projects have only a single Python file. I did scan through some other tutorials, the projects also have a single Python file.
  2. In the Dockerfile file, they both use RUN pip3 install -r requirements.txt to install required packages.

I would like to try building an image for a project which has more than a single module: most real projects would have more than one module. The code for Synology DS218: preparing Python 3.9 Beta compelete devepment environment. is fairly simple, and would be a good first try.

This is the repository https://github.com/behai-nguyen/app-demo for the code. For the above post, the tag is v1.0.0. It can be cloned with: 



git clone -b v1.0.0 https://github.com/behai-nguyen/app-demo.git
Enter fullscreen mode Exit fullscreen mode

Please note, all Docker builds discussed in this post've been out carried on Windows 10 Pro, using docker CLI version 20.10.12, build e91ed57.

To recap, the project layout for app-demo at tag v1.0.0 is: 



D:\app_demo\
|
|-- .env
|-- app.py
|-- setup.py
|
|-- src\
|   |
|   |-- app_demo\
|       |   
|       |-- __init__.py
|       |-- config.py
|
|-- venv\
Enter fullscreen mode Exit fullscreen mode

I did create virtualenv venv for this project in Windows 10.

Building using “RUN pip3 install -r requirements.txt” command

Please note, this image build step command renders setup.py not in use.

Generate the requirements.txt file with: 



D:\app_demo>venv\Scripts\pip.exe freeze > requirements.txt
Enter fullscreen mode Exit fullscreen mode

Then manually removed everything except for those packages specified in the section install_requires in the setup.py file. 



File D:\app_demo\requirements.txt
Enter fullscreen mode Exit fullscreen mode 


Flask==2.1.2
python-dotenv==0.20.0
Enter fullscreen mode Exit fullscreen mode 


File D:\app_demo\Dockerfile
Enter fullscreen mode Exit fullscreen mode 


# syntax=docker/dockerfile:1

FROM python:3.10.5-slim-buster

WORKDIR /app_demo

COPY requirements.txt requirements.txt
RUN pip3 install -r requirements.txt

COPY . .

CMD [ "python3", "-m" , "flask", "run", "--host=0.0.0.0" ]
Enter fullscreen mode Exit fullscreen mode 


File D:\app_demo\.dockerignore
Enter fullscreen mode Exit fullscreen mode 


__pycache__
*.pyc
*.pyo
*.pyd
.Python
venv
pip-log.txt
pip-delete-this-directory.txt
.tox
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.log
.git
.mypy_cache
.pytest_cache
.hypothesis
FlaskApp.wsgi
Working
Enter fullscreen mode Exit fullscreen mode

The root level of the project layout is now: 



D:\app_demo\
|
|-- .env
|-- app.py
|-- setup.py
|-- requirements.txt 
|-- Dockerfile
|-- .dockerignore
|
...
Enter fullscreen mode Exit fullscreen mode

The command to build: 



D:\app_demo>docker build --tag app-demo .
Enter fullscreen mode Exit fullscreen mode

The build runs successfully. To run the newly built image: 



D:\app_demo>docker run --publish 8000:5000 --rm app-demo
Enter fullscreen mode Exit fullscreen mode

It does not work, as can be seen in the screen capture below:

030-01-run-failed.png

The error is ModuleNotFoundError: No module named 'app_demo'.

These changes for this not-working-built can be cloned using: 



git clone -b v1.0.1 https://github.com/behai-nguyen/app-demo.git
Enter fullscreen mode Exit fullscreen mode

✿✿✿

Google search suggests that others also have experienced similar error. The suggestion is to use absolute import -- for example, please see Module not found error with Python in Docker.

I did try out absolute import, since the project is small, the changes are minuscule, and the resultant image does run. However, that does not seem right... I should not have to do that...

The changes to use absolute import can be cloned using: 



git clone -b v1.0.2 https://github.com/behai-nguyen/app-demo.git
Enter fullscreen mode Exit fullscreen mode

Basically, relevant import statements in: 



File D:\app_demo\app.py
File D:\app_demo\src\app_demo\__init__.py
Enter fullscreen mode Exit fullscreen mode

were prefixed with src. to become respectively: 



from src.app_demo import create_app
from src.app_demo.config import get_config
Enter fullscreen mode Exit fullscreen mode

Build and run, respectively, with: 



D:\app_demo>docker build --tag app-demo .
D:\app_demo>docker run --publish 8000:5000 --rm app-demo
Enter fullscreen mode Exit fullscreen mode

It runs successfully. http://localhost:8000 displays the expect output of Hello, World!

Building using “RUN pip3 install -e .” command

Please note for this image build step:

❶ Both: 



File D:\app_demo\app.py
File D:\app_demo\src\app_demo\__init__.py
Enter fullscreen mode Exit fullscreen mode

have their import statements reversed back to relative import: 



from app_demo import create_app
from app_demo.config import get_config
Enter fullscreen mode Exit fullscreen mode

That is, prefix src. added in the last build was removed.

❷ This image build step command renders requirements.txt obsolete.

✿✿✿

I was doing another Docker image for another Python project, I thought I would just try editable install RUN pip3 install -e . instead. I did think that I would have to change the source codes to absolute import anyway, so why not just use setup.py that's already in place. The build just went through with no problem. I ran it to get the first import failure... It does not fail!

It works! I still don't know why it works!

So I go back to this project, and hence this post. Changes are: 



File D:\app_demo\Dockerfile
Enter fullscreen mode Exit fullscreen mode 


# syntax=docker/dockerfile:1

FROM python:3.10.5-slim-buster

WORKDIR /app_demo

COPY . .

RUN /usr/local/bin/python -m pip install --upgrade pip && \
    pip3 install -e .

CMD [ "python3", "-m" , "flask", "run", "--host=0.0.0.0" ]
Enter fullscreen mode Exit fullscreen mode

requirements.txt is obsolete; it's added to D:\app_demo\.dockerignore. 



File D:\app_demo\.dockerignore
Enter fullscreen mode Exit fullscreen mode 


... 
requirements.txt
Enter fullscreen mode Exit fullscreen mode

Clone the new changes with the below command, please discard requirements.txt: 



git clone -b v1.0.3 https://github.com/behai-nguyen/app-demo.git
Enter fullscreen mode Exit fullscreen mode

Build and run, respectively, with: 



D:\app_demo>docker build --tag app-demo .
D:\app_demo>docker run --publish 8000:5000 --rm app-demo
Enter fullscreen mode Exit fullscreen mode

It runs successfully:

030-02-run-worked.png

http://localhost:8000 displays the expect output of Hello, World!

✿✿✿

I don't know why RUN pip3 install -e . works... Please tell me if you know, I would appreciate that very much.

Since I've made my decision to use setup.py, this Docker image build step just works out great. I'm happy with it. Thank you for reading and I hope you find this post useful somehow.

💖 💪 🙅 🚩
behainguyen
Be Hai Nguyen

Posted on July 22, 2022

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Python: Docker image build -- install required packages via requirements.txt vs editable install.
python Python: Docker image build -- install required packages via requirements.txt vs editable install.

July 22, 2022