In this two part series, we explore the world of IoT...or rather group of things.

We explain core concepts you need to get started with publishing messages across devices, and how to install the Python MQTT client (don't worry if you don't know what this is, I hope you will be the end of the piece).

Assumptions

• You have intermediate Python knowledge
• You can work with the Linux command line and follow prompts.

Sections

1. What is MQTT?
2. What is the Eclipse Paho Project?
3. What is the Eclipse Paho MQTT Python Client Library?
4. Paho MQTT Python Client Library Features
5. MQTT Origins
6. MQTT Basic Concepts
7. How to Install the Python MQTT Client
8. How to Connect to A Python MQTT Broker
9. Summary

What is MQTT?

MQTT (MQ Telemetry Transport) is one of the most popular IoT protocols - it is lightweight, scalable, has near real-time information delivery, and will work even in places with unreliable network connectivity.

Since a lot of IoT devices are battery-powered or have metered connections, they might not always be connected. With MQTT, messages can still be received or published when these devices reestablish the connection.

What is the Eclipse Paho Project?

No matter the MQTT use case, data needs to be exchanged between and among devices. The MQTT protocol makes this exchange almost seamless.

The Eclipse Paho project is one of the implementations of the MQTT protocol by the Eclipse Foundation (the organization behind popular open-source tools like the Eclipse IDE, and Jarkata EE)

The Eclipse Paho project uses different programming languages in its implementations, ranging from Python, Java, Javascript, Golang, C, C++, and Rust.

Our focus will be on MQTT’s Python implementation.

The Eclipse Paho MQTT Python Client Library

Eclipse Paho Client is a library that works with the MQTT publish-subscribe model to ensure that MQTT clients publish messages smoothly. It does this via helper functions and a client class. Supported MQTT versions include 5.0, 3.1.1, and 3.1

Paho MQTT Python Client Library Features

What features make the Paho MQTT Python Client a great option?

Last Will and Testament Messages (LWT)

For an MQTT client to connect to an MQTT broker, it needs to send a connection request. The connection request contains a CONNECT control packet with information that the broker will use for authorization and authentication, and to initiate a connection.

One of the flags in the CONNECT control packet is the Will. It determines whether the MQTT broker gets to store and publish the last will message associated with the last session. A will message is meant to be published once the MQTT client reestablishes a connection, in the event that the connection was interrupted or lost.

SSL/TLS

Security, even in IoT needs to be a priority. The SSL/TLS feature helps you ensure data confidentiality and integrity.

Automatic Reconnect

This feature ensures that the client can reconnect if the connection to the MQTT broker is severed.

Offline Buffering

This feature ‘stores’ messages when the recipient MQTT broker/client is disconnected.

Websocket Support

This feature allows for communication with brokers that allow the connection between client and server to persist, depending on the KeepAlive flag value in the CONNECT control packet.

Standard TCP Support

This feature allows for communication with brokers with TCP support.

Blocking API

This feature means that when the publisher requests the broker to send messages to the clients subscribed to a topic, execution only continues after successful delivery.

Non-Blocking API

This feature means that when the publisher requests the broker to send messages to the clients subscribed to a topic, execution can continue, and the clients can receive messages at different times.

Before looking at some paho mqtt examples, let’s understand MQTT’s origins and some basic concepts.

MQTT Origins

MQTT is an IoT protocol that enables the exchange of messages between devices. It uses a publish-subscribe model to achieve this.

MQTT works very similarly to the HTTP/HTTPS protocol (where a client sends in a request to a server and receives a response containing what they requested). MQTT is however extremely lightweight and has a low footprint.

MQTT was created by Andy Stanford-Clark (IBM) and Arlen Nipper (CTO at Cirrus Link) in 1999. It is now an OASIS standard, with v3.1.1 being an older ISO standard as well. Version 5 is the latest MQTT specification

MQTT Basic Concepts

It’s important to understand some MQTT basic concepts before you begin working with the paho mqtt examples. Let’s delve into them below:

The Publish-Subscribe Pattern/Model

It is also known as the pub-sub pattern. There are several devices that exchange data in this model which we will look at shortly.

In the pub-sub model, a publisher sends out messages to subscribers that have ‘subscribed’ to a particular topic.

The pub-sub model is decoupled (the publisher is not aware of the presence of the recipient subscribers).

The Broker

The broker or server is at the heart of the pub-sub model. It is the server equivalent in the HTTP/HTTPS protocol. All clients must connect to the server.

The broker filters all incoming messages and sends them out to the subscribers who have subscribed to specific topics.

The Publisher

The publisher is an MQTT client who sends messages to the broker for distribution to the subscribers.

The Subscriber

The subscriber is an MQTT client who receives messages from the broker for topics they are subscribed to.

Note:

Both publishers and subscribers need to establish a connection with the broker.

Sometimes an MQTT client can be both a publisher and a subscriber. For example, when a subscriber receives a message, it could send out a message to confirm the message’s receipt.This way, it becomes both a publisher and subscriber.

Topic

It is also known as a channel or subject. This is the area of interest that the subscriber subscribed to.

A Pub-Sub Pattern Demonstration

I will use a story from Richard Branson’s book, “Screw It, Let's Do It: Lessons in Life” to illustrate the pub-sub model.

In the book, Richard and his friend Per went on several hot air balloon adventures. They needed to know their altitude so as not to fly too high as to lack oxygen or too low so as to crash.

They used an altitude sensor (let’s call it sensor 1). For the sake of our demonstration, let’s assume that the altitude sensor had a Raspberry Pi 3 board.

The Raspberry Pi 3 board in the sensor will send out altitude information to various people:

• Richard and Per
• Air Traffic Control (in case of a crash)
• Maritime Patrol (in case of a crash)
• News station (in case of a crash)

The following diagram might make things even clearer:

Image source: "MQTT Essentials - A Lightweight IoT Protocol" by Gaston C. Hillar

The altitude sensor with the Raspberry Pi 3 board is the publisher (sensor1). It sends out messages to subscribers which could be anything from an Android tablet or iOS smartphone to a walkie-talkie in Richard’s or Per’s hand.

For the subscriber to receive information about the altitude, however, they need to have subscribed to the topic. In our example, the topic is ‘sensor1/altitude’.

The broker will receive the message and topic from the publisher and only send it to the subscribers subscribed to the ‘sensor1/altitude’ topic.

The news station will not receive the message because it is not subscribed to the ‘sensor1/altitude’ topic. The flying is a private event. The news station could be subscribed to another topic though, ‘sensor1/emergency’, and would only receive a message if there was an accident and Richard and Per’s flying had been made known to the public.

With that out of the way, then it will be easier for us to work with the MQTT protocol and the Paho Python MQTT Client.

How to Install the Python MQTT Client

There are several ways to install the Eclipse Paho Python MQTT Client.

Using Pip

Pip is Python’s package manager (Package Installer for Python in full)

Go to your terminal (Ctrl+Alt+T on Ubuntu) and enter the following command:

pip install paho-mqtt

You should see a ‘successfully installed’ message.

If you have any permission errors, you can use sudo

sudo pip install paho mqtt

You need to check the version to which Paho MQTT is attached if you have several Python versions.

pip –version

In my case, it was installed under Python 3.9. I will make a mental note of this when running Python paho-mqtt. I may have to specify the Python version for it to run as intended.

Installing paho-mqtt in a Virtual Environment

A virtual environment allows you to confine your project to a folder with all the necessary dependencies for it to run effectively. This way, you avoid conflicts that would result from using different versions of a package, for example.

The official Python documentation recommends the creation of virtual environments using the venv module.

Creating, activating, and working with a virtual environment

• Choose the directory in which to create your virtual environment.

• Create your virtual environment using the following command (make sure that your current working directory is the top-level directory containing the directory in which you wish to create your virtual environment)
  

python3 -m venv /path/to/directory/where/you/want/to/create/your/virtual/environment

• Activate your virtual environment (the following command works on a bash or zsh shell)

source /path/to/directory/where/you/want/to/create/your/virtual/environment/bin/activate

Note that you don’t need to activate your virtual environment. Activation only allows you to run the installed scripts without using full paths.

• Install Python paho-mqtt via pip

pip install paho-mqtt

How to Connect to a Python MQTT Broker

After installing the Paho MQTT Client, we need to use a broker to distribute the messages to the correct subscribers. We are going to install the broker first and then connect to it.

Installing the Mosquitto MQTT Broker

There are several ways to install the Mosquitto MQTT broker.

Installing from Source

Here you download the .tar file from the Mosquito downloads section on the official site or use the source code from the Eclipse Mosquitto git repo

Binary Installation

In this case, you can use a package manager like pip (Linux) or brew (MacOS) depending on your Operating System and/or distro.

Mosquitto downloads section

For our illustration, we will use Linux Ubuntu 20.04

• Enter the following command to install the Mosquitto MQTT broker

sudo apt-add-repository ppa:mosquitto-dev/mosquitto-ppa

• Enter your password

• Follow the prompts

• Enter sudo apt-get update

• Install the Ubuntu broker package
  

sudo apt-get install mosquitto

• Follow the prompts

• Choose 'y'

• Press ‘enter’

• Install the Mosquitto client packages that run the message publication commands and topic subscription via the following command:
  

sudo apt-get install mosquitto-clients

• Check the recently installed mosquitto service status

sudo service mosquitto status

• You should see a message like… ‘starting MQTT Broker…’

• Confirm that the Mosquitto server is listening via the default port: 1883
  

netstat -an | grep 1883

Summary

We have discussed core concepts you need to get started with publishing messages across devices, and how to install the Python MQTT client.

In Part 2, we discuss how to publish messages, subscribing to topics, and how to disconnect from the broker.