Django & Celery - Open-Source Tasks Manager

sm0ke

Sm0ke

Posted on October 12, 2022

Django & Celery - Open-Source Tasks Manager

Hello!

This article presents an open-source Tasks Manager powered by Django and Celery. The product can be used to execute background tasks that are usually time-consuming in full control: start, cancel the execution, view output, and runtime logs once the tasks are finished. Being released under the MIT license, the source can be used in commercial projects or eLearning activities. Thanks for reading!


✨ Product Highlights

This project implements a mechanism that allows to start/cancel tasks on demand using a simple UI - Here are the basic features provided in this first version:

  • ✅ Framework: Django
  • ✅ Async Tasks Management via Celery
  • ✅ Actions: Create/Cancel
  • ✅ UI Access to Runtime logs and Output
  • ✅ Types of tasks: Scripts RUNNER, and Users Print
  • ✅ Bootstrap 5 Design, Dark Mode

Django & Celery - Registered Tasks

Once a task is started, the current state is reflected immediately in the UI. This behavior is visible in the UI via the UI controls that suggest the state of the task and the Cancelling option.

Django & Celery - Task Is Started


All running tasks are cancelable, and this feature might be useful when a background task runs more than expected.

For finished tasks, the users are able to check the task output directly in the UI and visualize the RUNTIME log in case more execution details are required.

Django & Celery - Task Runtime LOG


✨ How to use the product

The background tasks processing requires a running Redis service and the address of the service should be configured in the project settings (dedicated section):

#############################################################
# Celery configurations
# https://docs.celeryq.dev/en/stable/django/first-steps-with-django.html

CELERY_BROKER_URL         = os.environ.get("CELERY_BROKER", "redis://localhost:6379")
CELERY_RESULT_BACKEND     = os.environ.get("CELERY_BROKER", "redis://localhost:6379")
CELERY_TASK_TRACK_STARTED = True
CELERY_TASK_TIME_LIMIT    = 30 * 60
CELERY_CACHE_BACKEND      = "django-cache"
CELERY_RESULT_BACKEND     = "django-db"
CELERY_RESULT_EXTENDED    = True
CELERY_RESULT_EXPIRES     = 60*60*24*30 # Results expire after 1 month
CELERY_ACCEPT_CONTENT     = ["json"]
CELERY_TASK_SERIALIZER    = 'json'
CELERY_RESULT_SERIALIZER  = 'json'

#############################################################
#############################################################
Enter fullscreen mode Exit fullscreen mode

If we assume that Redis is up & running, here are the steps to start the product in a local environment:


👉 Step #1 - Clone the sources from the public repository

$ git clone https://github.com/app-generator/sample-django-celery.git
$ cd sample-django-celery
Enter fullscreen mode Exit fullscreen mode

👉 Step #2 - Install the modules using a virtual environment

$ virtualenv env
$ source env/bin/activate
$ pip3 install -r requirements.txt
Enter fullscreen mode Exit fullscreen mode

👉 Step #3 - Migrate the database

$ python manage.py makemigrations
$ python manage.py migrate
Enter fullscreen mode Exit fullscreen mode

👉 Step #4 - Start the project

$ python manage.py runserver
Enter fullscreen mode Exit fullscreen mode

Using a separate terminal window, the following command starts the Celery manager:

👉 Step #5 - Activate the VENV

$ source env/bin/activate
Enter fullscreen mode Exit fullscreen mode

👉 Step #6 - Start the manager

$ celery --app=core.celery.app worker --loglevel=info 
Enter fullscreen mode Exit fullscreen mode

At this point, we should be able to execute tasks using a SuperUser Account. Ordinary users are able only to check out the tasks output and runtime logs.

Django & Celery - Tasks List


✨ Video Presentation

A short demonstration that summarizes all the above features can be found on YouTube.


Thanks for reading! For more resources, feel free to access:

💖 💪 🙅 🚩
sm0ke
Sm0ke

Posted on October 12, 2022

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related