Meir Gabay
Posted on November 5, 2020
Goal
Covering the possible ways to execute a Python module (script) from the command-line (terminal).
Requirements
- You are familiar with Packages and Modules
- It would be easier to understand the examples if you run the examples by yourself - use this project as a template
Getting Started
We'll go over the possible way to execute scripts and modules from the command-line
Executing Scripts And Modules
In the following examples, all the modules are executed from the project's root directory (top-level package). This is important since the root dir has access to all the packages (sub-directories) in the project.
Executing the main.py script and appy as a module with src/appy/__main__.py
# Executing as a script - works as long as there are no relative imports of parent packages
(python-project) $ python main.py
My Path: python-project/main.py
Created the file: /Users/meirgabay/python-project/meirg-ascii.txt
Insert your name: willy wonka
Hello Willy Wonka, here's the cat fact of the day:
Jaguars are the only big cats that don't roar.
# Executing as a module - works because of the __main__.py file
(python-project) $ python -m src.appy
My Path: python-project/src/appy/__main__.py
Created the file: /Users/meirgabay/python-project/meirg-ascii.txt
Insert your name: willy wonka
Hello Willy Wonka, here's the cat fact of the day:
A cat has more bones than a human being; humans have 206 and the cat has 230 bones.
Executing the module src/appy/core/app.py from the root directory
# Contains relative imports - `..utils.message`
# Excuting as a script - causes issues with relative imports of parent packages
(python-project) $ python src/appy/core/app.py
Traceback (most recent call last):
File "src/appy/core/app.py", line 1, in <module>
from ..utils import message, img_ascii
ImportError: attempted relative import with no known parent package
# Excuting as a module - works perfect with relative imports
(python-project) $ python -m src.appy.core.app
My Path: python-project/src/appy/core/app.py
Created the file: /Users/meirgabay/python-project/meirg-ascii.txt
Insert your name: willy wonka
Hello Willy Wonka, here's the cat fact of the day:
Siamese kittens are born white because of the heat inside the mother's uterus before birth. This heat keeps the kittens' hair from darkening on the points.
Executing the module src/appy/utils/message.py from the root dir
# Doesn't contain relative imports
# Executing as a script - no errors since there are no relative imports
(python-project) $ python src/appy/utils/message.py
My Path: python-project/src/appy/utils/message.py
# Executing as a module - works as expected
(python-project) $ python -m src.appy.utils.message
python-project/src/appy/utils/message.py
If the PWD is a subdirectory of the project, such as python-project/src/appy, an attempt to execute a module which contains relative imports, will raise the exception below. Remember, your PWD should always be the project's root directory, in this case, it's python-project.
# PWD is `src/appy`
# Execute as a module - issues when running from a subdirectory (not from root dir)
(python-project/src/appy) $ python -m core.app
Traceback (most recent call last):
File "/Users/meirgabay/.pyenv/versions/3.8.2/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 193, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/Users/meirgabay/.pyenv/versions/3.8.2/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/Users/meirgabay/python-project/src/appy/core/app.py", line 1, in <module>
from ..utils import message, img_ascii
ValueError: attempted relative import beyond top-level package
It doesn't happen when invoking message, since message doesn't use relative imports
# Execute as a script - works because there are no relative imports to parent packages, similar to main.py
(python-project/src/appy) $ python utils/message.py
My Path: python-project/src/appy/utils/message.py
# Execute as a module - works as expected, no imports of parent packages
(python-project/src/appy) $ python -m utils.message
My Path: python-project/src/appy/utils/message.py
Invoke a function from the command-line
Trying to invoke a function from the terminal, such as appy.core.app.main(), will raise the ModuleNotFound exception. A package must be imported before invoking one of its functions.
# Execute as a module - must import the function before invoking it
(python-project) $ python -m src.appy.core.app.main
/Users/meirgabay/.pyenv/versions/3.8.2/Python.framework/Versions/3.8/bin/python: Error while finding module specification for 'src.appy.core.app.main' (ModuleNotFoundError: __path__ attribute not found on 'src.appy.core.app' while trying to find 'src.appy.core.app.main')
Since you can't invoke main() directly from the terminal, calling it from the if __main__ block enables executing it from the terminal. It's possible to pass arguments, but it's a bit ugly, read the docs to learn how. The following example attempts to execute the module appy.core.app, which in turn call its if __main__ block
# Execute as a module - works due to the `if __main__` block
(python-project) $ python -m src.appy.core.app
My Path: python-project/src/appy/core/app.py
Created the file: /Users/meirgabay/python-project/meirg-ascii.txt
Insert your name: willy wonka
Hello Willy Wonka, here's the cat fact of the day:
Cats sleep 16 to 18 hours per day. When cats are asleep, they are still alert to incoming stimuli. If you poke the tail of a sleeping cat, it will respond accordingly.
Invoking a function from the terminal is also possible by using the -c flag. Surprise, it's possible to pass arguments in a more intuitive way, for example src.app.main(my_arg1, my_arg2)
# Execute inline script - providing the "-c" flag with the command as a string, the semi-colon ";" means break-row
(python-project) $ python -c "import src.appy.core.app as app; app.main()"
Created the file: /Users/meirgabay/python-project/meirg-ascii.txt
Insert your name: willy wonka
Hello Willy Wonka, here's the cat fact of the day:
Cats can be right-pawed or left-pawed.
What are the available command-line flags in Python?
- In this tutorial, we used both
-cand-mflags - Read the docs to learn about more flags - Using cmdline
Why is it possible to execute python -m src.appy?
The src/appy/__main__.py file acts like the if __main__ code snippet, but on packages. This enables the appy package to be executed with python -m or with runpy
# Execute as a module
(python-project) $ python -m src.appy
My Path: python-project/src/appy/__main__.py
Created the file: /Users/meirgabay/python-project/meirg-ascii.txt
Insert your name: willy wonka
Hello Willy Wonka, here's the cat fact of the day:
One reason that kittens sleep so much is because a growth hormone is released only during sleep.
What's runpy and why do you use it in main.py?
The runpy package provides the ability to run modules from a module (Python script). This package is shipped with Python so there's no need to install it.
import runpy
def main():
appy_package = runpy.run_module(
mod_name="src.appy", init_globals=globals())
appy_package['message'].script_path(__file__)
appy_package['main']()
if __name__ == "__main__":
main()
What's globals()?
The official definition from the docs
Return a dictionary representing the current global symbol table. This is always the dictionary of the current module (inside a function or method, this is the module where it is defined, not the module from which it is called).
(python-project) $ python
Python 3.8.2 (default, Jun 30 2020, 19:04:41)
[Clang 11.0.3 (clang-1103.0.32.59)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> globals()
{'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <class '_frozen_importlib.BuiltinImporter'>, '__spec__': None, '__annotations__': {}, '__builtins__': <module 'builtins' (built-in)>}
This blog post is part of the Python Project series, and is based on this GitHub repository - unfor19/python-project.
The GitHub repo includes an example of a Python project, and Wiki Pages that describe the necessary steps for developing, creating and distributing a Python package.
Originally published at github.com/unfor19/python-project on 3 November, 2020
Posted on November 5, 2020
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related
November 13, 2024