TheLazy.Dev
Portfolio GitHub

Blog

python |tutorial |bot |webscraping

How to Build a Price Monitoring Slack Bot 🤑

maxhumber

Max Humber

Posted on September 3, 2020

How to Build a Price Monitoring Slack Bot 🤑

I just got a new Switch (because what else is there to do these days?!) And I really wanna get the new Paper Mario game. Except, I'm not keen on paying full pop for the title... So. I built a price monitoring Slack Bot to tell me if it's on sale! Here's how you can build one too~

First, install gazpacho, hickory, and slackclient:

pip install gazpacho hickory slackclient
Enter fullscreen mode Exit fullscreen mode

Identify the page you want to scrape:

BestBuy

Grab the URL, open a Text Editor (I like Atom), create a python file called mario.py and import gazpacho:

from gazpacho import get, Soup
Enter fullscreen mode Exit fullscreen mode

Download the html with the get function and stuff it in a Soup wrapper:

url = "https://www.bestbuy.ca/en-ca/product/paper-mario-the-origami-king-switch/14643866"
html = get(url)
soup = Soup(html)
Enter fullscreen mode Exit fullscreen mode

Figure out how the price is being rendered on the page by right clicking on it and opening "Inspect Element:

Inspect Element

Note the specific tag and element attributes:

HTML

In this case the relevant block of html on the page is:

<div class="price_FHDfG large_3aP7Z " aria-hidden="true">
  $79
  <span class="rightEndPrice_6y_hS">99</span>
</div>
Enter fullscreen mode Exit fullscreen mode

Use the find method with partial matching (strict=False) enabled to extract the price and format it as a float:

price = soup.find("div", {"class": "price"}, strict=False)
lhs = price.text
rhs = price.find("span", {'class': 'rightEndPrice'}).text
price = float(f"{lhs}.{rhs}".replace('$', ""))
Enter fullscreen mode Exit fullscreen mode

Might as well grab the title too:

title = soup.find('h1', {'class': 'productName'}).text
Enter fullscreen mode Exit fullscreen mode

Print out both to confirm that everything is working:

print(title, price)
# Paper Mario: The Origami King (Switch) 79.99
Enter fullscreen mode Exit fullscreen mode

Your entire script should look like this:

# mario.py
from gazpacho import get, Soup

url = "https://www.bestbuy.ca/en-ca/product/paper-mario-the-origami-king-switch/14643866"
html = get(url)
soup = Soup(html)

title = soup.find('h1', {'class': 'productName'}).text
price = soup.find("div", {"class": "price"})
lhs = price.text
rhs = price.find("span", {'class': 'rightEndPrice'}).text
price = float(f"{lhs}.{rhs}".replace('$', ""))

print(title, price)
Enter fullscreen mode Exit fullscreen mode

To schedule this scraper and turn it into a Slack Bot go to https://api.slack.com/apps and click Create New App:

Slack API

Find and click on OAuth & Permissions:

OAuth

Search for the chat:public.write scope and add it:

Scopes

Scroll back up to the top of the page and install the app into your Slack Workspace:

Install

Copy the generated token to the mario.py file as a string:

Alt Text

token = "xoxb-00000000-0000000000-00000000000"
Enter fullscreen mode Exit fullscreen mode

Import slack and instantiate an instance of WebClient:

from slack import WebClient
client = WebClient(token=token)
Enter fullscreen mode Exit fullscreen mode

Send a message to a Slack channel with the chat_postMessage method:


client.chat_postMessage(channel="mario", text=f"{price} - {title}")
Enter fullscreen mode Exit fullscreen mode

Re-factor everything to make it look like:

# mario.py

from gazpacho import get, Soup
from slack import WebClient

token = "xoxb-00000000-00000000-000000000000000"
client = WebClient(token=token)

url = "https://www.bestbuy.ca/en-ca/product/paper-mario-the-origami-king-switch/14643866"
html = get(url)
soup = Soup(html)

title = soup.find('h1', {'class': 'productName'}).text
price = soup.find("div", {"class": "price"})
lhs = price.text
rhs = price.find("span", {'class': 'rightEndPrice'}).text
price = float(f"{lhs}.{rhs}".replace('$', ""))

client.chat_postMessage(channel="mario", text=f"${price} - {title}")
Enter fullscreen mode Exit fullscreen mode

You really shouldn't expose your token as plaintext but, whatever...

YOLO

Run the script at the command line to make sure it works:

python mario.py
Enter fullscreen mode Exit fullscreen mode

Scheduling this script to every 30 seconds with hickory is dead simple. Just use hickory schedule at the command line:

hickory schedule mario.py --every=30s
Enter fullscreen mode Exit fullscreen mode

To check the status of the schedule run:

hickory status
# ID      FILE         STATE    RUNS  INTERVAL
# ea6b74  mario.py     waiting  0     30
Enter fullscreen mode Exit fullscreen mode

And to kill the schedule:

hickory kill mario.py
Enter fullscreen mode Exit fullscreen mode

Scheduling this Price Monitoring Slack Bot to execute every day at 9:00am is probably more realistic:

hickory schedule mario.py --every=@9:00am
Enter fullscreen mode Exit fullscreen mode

And that's it! 🤑

If you've built something cool with gazpacho and/or hickory get in touch, I'd love to hear about it!

💖 💪 🙅 🚩
maxhumber
Max Humber

Posted on September 3, 2020

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

How to Build a Price Monitoring Slack Bot 🤑
python How to Build a Price Monitoring Slack Bot 🤑

September 3, 2020