Recently I had a requirement of implementing a web scraping tool to identify changes in a website. It was the virtual learning environment website of my university where I wanted to create a desktop notification whenever our examination results were uploaded to the website. I started by writing a Python script that checks the website and creates a notification on Windows if updates are detected. I used the Python Beautiful Soup library to parse the HTML content of the web page and the Windows 10 Toast Notifications library to generate Windows 10 toast notifications. These tools made the implementation pretty simple.

The next phase was to run the script every 10 minutes and my initial thought was to use time.sleep() while the script was running in an infinite loop. That's when I started to realize that even though this approach solves the problem, using time.sleep() does not mean that the code is executed every 10 minutes. What it does is put a 10 minute gap between executions. For a simple expendable application(like what I'm implementing) this is not an issue but for mission-critical applications or streaming applications using time.sleep() for scheduling might have adverse effects. So I decided to use Windows Task Scheduler to run my Python script every 10 minutes.

Steps to schedule a Python script using the Windows Task Scheduler

Step 1 - Create a batch file to run the Python script

Open Notepad, add the following syntax and save it with the .bat extension.

start "" "path_to_python.exe" "path_to_python_script"

Example:

start "" "C:\Python38\python.exe" "C:\Users\Tharinda\Desktop\web_scraper\main.py"

Using start makes sure the cmd is closed after the execution. The first "" is the title that should be used with start and in this case, the title is left blank.

After double-clicking on the .bat file the python script should run if the file is configured properly.

Step 2 - Create a task in Windows Task Scheduler

Open the start menu and search for Task Scheduler.

• In the Actions tab select Create Task

• Give a name for the task

• Switch to the Triggers tab and press New to create a new trigger.

• Configure the trigger to suit your needs. In my case, the trigger is scheduled to run daily repeating the task every 10 minutes.

• Switch to the Actions tab and press New to create a new Action.

• Under Program/Script add the batch file that was created in step 1

Add Conditions and change the Settings depending on your requirement and press OK to save the task. You can refer the Task Scheduler documentation to learn more about the Windows Task Scheduler.

That is pretty much it. Your task should run as intended.

I hope this helps. Cheers!