A few lesser-known but pretty useful Python concepts

Context Managers
Generators and Generator Expressions
Function Decorators
Namedtuples
Coroutines
Operator Overloading
Monkey Patching
Type Hints and Annotations
Metaclasses
Data Classes
Context Variables
Callable Objects
Extended Iterable Unpacking
Multiple Inheritance and Method Resolution Order (MRO)

A. Context Managers:

Context managers allow you to allocate and release resources when needed, such as opening and closing files, acquiring and releasing locks, etc. They can be implemented using the with statement and the contextlib module.

Examples

File Handling

with open('file.txt', 'r') as file:
    data = file.read()
    # Perform operations on the file
# File automatically closed outside the 'with' block

Lock Acquisition and Release

import threading

lock = threading.Lock()
with lock:
    # Perform thread-safe operations
# Lock automatically released outside the 'with' block

Database Connection

import sqlite3

with sqlite3.connect('database.db') as connection:
    cursor = connection.cursor()
    # Perform database operations
# Connection automatically closed outside the 'with' block

Timing Execution

import time

class Timer:
    def __enter__(self):
        self.start_time = time.time()
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        elapsed_time = time.time() - self.start_time
        print(f"Execution time: {elapsed_time} seconds")

with Timer():
    # Code to measure execution time
# Timer context manager prints the execution time

Creating a custom context manager

class CustomContextManager:
    def __enter__(self):
        # Code to run when entering the 'with' block
        print("Entering the 'with' block")
        # You can optionally return an object or set up resources

    def __exit__(self, exc_type, exc_val, exc_tb):
        # Code to run when exiting the 'with' block
        print("Exiting the 'with' block")
        # Perform any cleanup actions or handle exceptions if necessary

# Usage example:
with CustomContextManager():
    # Code inside the 'with' block
    print("Inside the 'with' block")

# Output:
# Entering the 'with' block
# Inside the 'with' block
# Exiting the 'with' block

B. Generators and Generator Expressions:

Generators are functions that generate values on the fly, allowing you to iterate over a sequence of values without creating the entire sequence in memory. Generator expressions are similar to list comprehensions but return a generator object instead of a list.

Generator Function

def countdown(n):
    while n > 0:
        yield n
        n -= 1

# Using the generator function
for num in countdown(5):
    print(num)

# Output:
# 5
# 4
# 3
# 2
# 1

Generator Expression

# Using a generator expression to calculate the squares of numbers
squares = (x ** 2 for x in range(1, 6))

# Accessing the values from the generator expression
for square in squares:
    print(square)

# Output:
# 1
# 4
# 9
# 16
# 25

Infinite Generator

def infinite_sequence():
    num = 0
    while True:
        yield num
        num += 1

# Using the infinite generator
for i in infinite_sequence():
    if i > 10:
        break
    print(i)

# Output:
# 0
# 1
# 2
# 3
# 4
# 5
# 6
# 7
# 8
# 9
# 10

C. Function Decorators:

Decorators are a way to modify the behavior of functions or classes by wrapping them with another function. They are denoted by the @ symbol and can be used for tasks like logging, timing, and caching.

Logging Decorator: This decorator logs the name of the decorated function before and after its execution

def log_decorator(func):
    def wrapper(*args, **kwargs):
        print(f'Calling {func.__name__}...')
        result = func(*args, **kwargs)
        print(f'{func.__name__} called.')
        return result
    return wrapper

@log_decorator
def add_numbers(a, b):
    return a + b

result = add_numbers(3, 5)
print(result)

# Output:
# Calling add_numbers...
# add_numbers called.
# 8

Timing Decorator: This decorator measures the execution time of the decorated function

import time

def time_decorator(func):
    def wrapper(*args, **kwargs):
        start_time = time.time()
        result = func(*args, **kwargs)
        end_time = time.time()
        execution_time = end_time - start_time
        print(f'{func.__name__} executed in {execution_time} seconds.')
        return result
    return wrapper

@time_decorator
def factorial(n):
    if n == 0 or n == 1:
        return 1
    else:
        return n * factorial(n - 1)

result = factorial(5)
print(result)  # Output: 120

# Output:
# factorial executed in 9.5367431640625e-07 seconds.
# factorial executed in 7.605552673339844e-05 seconds.
# factorial executed in 9.322166442871094e-05 seconds.
# factorial executed in 0.00010275840759277344 seconds.
# factorial executed in 0.00011801719665527344 seconds.
# 120

Authorization Decorator: This decorator checks if the user is authorized to access the decorated function

def is_user_authorized():
    return False

def authorize_decorator(func):
    def wrapper(*args, **kwargs):
        if is_user_authorized():
            return func(*args, **kwargs)
        else:
            raise PermissionError('User is not authorized to access this function.')
    return wrapper

@authorize_decorator
def sensitive_operation():
    return 'Sensitive data'

result = sensitive_operation()
print(result)

# Output:
# Traceback (most recent call last):
#   File "/Users/dev1/python/app.py", line 16, in <module>
#     result = sensitive_operation()
#   File "/Users/dev1/mygitlab/python/app.py", line 9, in wrapper
#     raise PermissionError('User is not authorized to access this function.')
# PermissionError: User is not authorized to access this function.

D. Namedtuples:

Namedtuples are lightweight data structures that are similar to tuples but have named fields. They provide a convenient way to define simple classes without writing a custom class definition. They are commonly used in scenarios where lightweight data containers are required.

Creating a Namedtuple

from collections import namedtuple

# Define a namedtuple called 'Point' with fields 'x' and 'y'
Point = namedtuple('Point', ['x', 'y'])

# Create an instance of Point
p = Point(2, 3)

# Access the fields using dot notation
print(p.x)  # Output: 2
print(p.y)  # Output: 3

Namedtuple as a Return Type

from collections import namedtuple

# Define a namedtuple called 'Person' with fields 'name', 'age', and 'city'
Person = namedtuple('Person', ['name', 'age', 'city'])

# Function that returns a Person namedtuple
def get_person():
    return Person('John', 25, 'New York')

# Call the function and access the fields of the returned namedtuple
person = get_person()
print(person.name)  # Output: John
print(person.age)   # Output: 25
print(person.city)  # Output: New York

Unpacking a Namedtuple

from collections import namedtuple

# Define a namedtuple called 'Color' with fields 'red', 'green', and 'blue'
Color = namedtuple('Color', ['red', 'green', 'blue'])

# Create an instance of Color
color = Color(255, 128, 0)

# Unpack the values into separate variables
red, green, blue = color

print(red)    # Output: 255
print(green)  # Output: 128
print(blue)   # Output: 0

E. Coroutines:

Coroutines are functions that can pause their execution and yield control back to the caller while maintaining their state. They are used for asynchronous programming and are a powerful tool for managing concurrency and asynchronous programming, allowing for more efficient and flexible code execution.

Simple Coroutine

def coroutine_example():
    while True:
        x = yield
        print('Received:', x)

coroutine = coroutine_example()
next(coroutine)  # Initialize the coroutine
coroutine.send(10)  # Send a value to the coroutine
coroutine.send('Hello')  # Send another value to the coroutine
coroutine.close()   # close it

# Output:
# Received: 10
# Received: Hello

Coroutine with Producer and Consumer

def producer(coroutine):
    for i in range(5):
        print('Producing:', i)
        coroutine.send(i)
    coroutine.close()

def consumer():
    while True:
        x = yield
        print('Consumed:', x)

coroutine = consumer()
next(coroutine)  # Initialize the coroutine
producer(coroutine)

# Output:
# Producing: 0
# Consumed: 0
# Producing: 1
# Consumed: 1
# Producing: 2
# Consumed: 2
# Producing: 3
# Consumed: 3
# Producing: 4
# Consumed: 4

Coroutine Chaining

def coroutine1():
    while True:
        x = yield
        print('Coroutine 1:', x)

def coroutine2():
    while True:
        x = yield
        print('Coroutine 2:', x)

coroutine = coroutine1()
next(coroutine)  # Initialize the first coroutine
coroutine2()  # Initialize the second coroutine
coroutine.send(10)  # Send a value to the first coroutine
coroutine.send('Hello')  # Send another value to the first coroutine

# Output:
# Coroutine 1: 10
# Coroutine 1: Hello

F. Operator Overloading:

Python allows you to redefine the behavior of operators for your custom classes by implementing special methods such as __add__, __sub__, __mul__, etc. This concept is known as operator overloading. You can overload various operators such as arithmetic operators, comparison operators, and more. Operator overloading allows you to customize the behavior of objects to make your code more expressive and intuitive

Arithmetic Operators

class Vector:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def __add__(self, other):
        return Vector(self.x + other.x, self.y + other.y)

    def __sub__(self, other):
        return Vector(self.x - other.x, self.y - other.y)

    def __mul__(self, scalar):
        return Vector(self.x * scalar, self.y * scalar)

    def __str__(self):
        return f"x: {self.x}, y: {self.y}"

# Usage:
v1 = Vector(2, 3)
v2 = Vector(4, 5)

v3 = v1 + v2  # Addition
v4 = v1 - v2  # Subtraction
v5 = v1 * 2   # Scalar multiplication

print(v3)
print(v4)
print(v5)

# Output:
# x: 6, y: 8
# x: -2, y: -2
# x: 4, y: 6

Comparison Operators

class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def __eq__(self, other):
        return self.x == other.x and self.y == other.y

    def __lt__(self, other):
        return self.x < other.x and self.y < other.y

# Usage:
p1 = Point(2, 3)
p2 = Point(4, 5)

print(p1 == p2)  # Equality
print(p1 < p2)   # Less than

# Output:
# False
# True

String Representation

class Book:
    def __init__(self, title, author):
        self.title = title
        self.author = author

    def __str__(self):
        return f"{self.title} by {self.author}"

# Usage:
book = Book("Python Programming", "John Smith")
print(book)  # String representation

# Output:
# Python Programming by John Smith

G. Monkey Patching:

Monkey patching refers to the ability to modify or extend the behavior of existing code at runtime. In Python, you can dynamically modify classes, objects, or modules by adding, replacing, or deleting attributes or methods.

Adding a Method to a Class

class MyClass:
    def __init__(self, name):
        self.name = name

def say_hello(self):
    print(f"Hello, {self.name}!")

# Monkey patching
MyClass.say_hello = say_hello

# Creating an instance and calling the patched method
obj = MyClass("Alice")
obj.say_hello()

# Output:
# Hello, Alice!

Modifying an Existing Method

class MyClass:
    def greeting(self):
        return "Hello!"

# Monkey patching
def modified_greeting(self):
    return "Hola!"

MyClass.greeting = modified_greeting

# Creating an instance and calling the modified method
obj = MyClass()
print(obj.greeting())

# Output:
# Hola!

Adding a Function to a Module

def multiply(a, b):
    return a * b

# Monkey patching
import math
math.multiply = multiply

# Calling the patched function
result = math.multiply(5, 6)
print(result)

# Output:
# 30

H. Type Hints and Annotations:

Type hints are a way to statically declare the expected types of variables, arguments, and return values in Python code. They are not enforced at runtime but can be used by static analysis tools to catch potential type-related errors.

Variable type hints:

def add(a: int, b: int) -> int:
    return a + b

Class annotations:

class Person:
    def __init__(self, name: str, age: int):
        self.name = name
        self.age = age

Type hints for collections

from typing import List, Tuple

def process(data: List[Tuple[str, str]]) -> List[str]:
    result: List[str] = []

    for item in data:
        result.append(item[0] * item[1])

    return result

I. Metaclasses

Metaclasses in Python provide a way to define the behavior and structure of classes themselves. They allow you to customize the creation and initialization of classes.

Metaclasses provide a powerful mechanism for customizing class creation and behavior, but they should be used sparingly and only when necessary, as they can make the code more complex and harder to understand.

Creating a Singleton Metaclass

class SingletonMeta(type):
    _instances = {}

    def __call__(cls, *args, **kwargs):
        if cls not in cls._instances:
            cls._instances[cls] = super().__call__(*args, **kwargs)
        return cls._instances[cls]


class SingletonClass(metaclass=SingletonMeta):
    def __init__(self):
        print("Initializing SingletonClass")


# Usage
instance1 = SingletonClass()
instance2 = SingletonClass()
print(instance1 is instance2)

# Output:
# Initializing SingletonClass
# True

Creating an Attribute Validation Metaclass

class ValidationMeta(type):
    def __new__(cls, name, bases, attrs):
        for name, value in attrs.items():
            if name.startswith("_"):
                continue

            if not isinstance(value, (int, float)):
                raise TypeError(f"Attribute '{name}' must be numeric.")

        return super().__new__(cls, name, bases, attrs)


class MyClass(metaclass=ValidationMeta):
    x = 10
    y = "test"

# Output:
# Traceback (most recent call last):
#   File "/Users/dev1/python/learnings/app.py", line 13, in <module>
#     class MyClass(metaclass=ValidationMeta):
#   File "/Users/dev1/python/learnings/app.py", line 8, in __new__
#     raise TypeError(f"Attribute '{name}' must be numeric.")
# TypeError: Attribute 'y' must be numeric

Registering Subclasses with a Metaclass

class PluginMeta(type):
    def __init__(cls, name, bases, attrs):
        if not hasattr(cls, "plugins"):
            cls.plugins = []
        else:
            cls.plugins.append(cls)


class PluginBase(metaclass=PluginMeta):
    pass


class Plugin1(PluginBase):
    pass


class Plugin2(PluginBase):
    pass


# Usage
print(PluginBase.plugins)

# Output
# [<class '__main__.Plugin1'>, <class '__main__.Plugin2'>]

J. Data Classes

Python's data classes are a convenient way to define classes that primarily hold data, similar to structs in other programming languages. They provide several benefits, such as automatic generation of common methods like __init__, __repr__, and __eq__. Data classes provide additional features like type hints, default values, ordering, and more.

Basic Data Class

from dataclasses import dataclass

@dataclass
class Point:
    x: float
    y: float
    z: float

p = Point(1.0, 2.0, 3.0)
q = Point(1.0, 2.0, 3.0)

print(p.x)
print(p.y)
print(p.z)

print(p)
print(p == q)

# Output
# 1.0
# 2.0
# 3.0
# Point(x=1.0, y=2.0, z=3.0)
# True

Data Class with Default Values

from dataclasses import dataclass

@dataclass
class Person:
    name: str
    age: int = 0
    profession: str = "Unknown"

# Create an instance of the Person class
person1 = Person("Alice", 25)
person2 = Person("Bob", profession="Engineer")

# Access the fields of the instances
print(person1.name, person1.age, person1.profession)
print(person2.name, person2.age, person2.profession)

# Output:
# Alice 25 Unknown
# Bob 0 Engineer

Inheritance with Data Classes

from dataclasses import dataclass

@dataclass
class Person:
    name: str
    age: int

@dataclass
class Employee(Person):
    company: str

# Create an instance of the Employee class
employee = Employee("Alice", 30, "Acme Corporation")

# Access the fields of the instance
print(employee.name)
print(employee.age)
print(employee.company)

# Output:
# Alice
# 30
# Acme Corporation

K. Context Variables

Introduced in Python 3.7, context variables allow you to create variables that retain their values within a context, even if the context is asynchronous or multi-threaded. They are useful for propagating values across functions without passing them explicitly as arguments.

Using contextvars module

import contextvars

user_id = contextvars.ContextVar("user_id", default=None)

def process_request(request):
    user_id.set(request.user_id)
    process_data()


def process_data():
    uid = user_id.get()
    print(f"processing for user: {uid}")

Using threading.local for thread-local context

import threading

context = threading.local()

def process_request(request):
    context.user_id = request.user_id
    process_data()

def process_data():
    uid = context.user_id
    print(f"processing for user: {uid}")

Using contextlib.ContextDecorator for context-based behavior

from contextlib import ContextDecorator
import time

class TimingContext(ContextDecorator):
    def __enter__(self):
        self.start_time = time.time()

    def __exit__(self, exc_type, exc_val, exc_tb):
        elapsed_time = time.time() - self.start_time
        print("Elapsed time:", elapsed_time)

@TimingContext()
def process_data():
    # Perform some time-consuming operation
    time.sleep(3)
    print("Data processing complete.")

process_data()

# Output:
# Data processing complete.
# Elapsed time: 3.0054383277893066

L. Callable Objects

In Python, any object that can be called as a function is considered callable. This includes functions, methods, classes (which create instances when called), and objects implementing the __call__ method. You can use callable objects to create more flexible and dynamic code.

Classes with __call__ method: Classes that define the __call__ method can be called like functions

class Multiply:
    def __call__(self, a, b):
        return a * b

multiply = Multiply()
result = multiply(2, 3)
print(result)

# Output:
# 6

Built-in callable objects: Some built-in objects in Python are callable, such as int, str, list, dict, etc.

result = str(42)
print(result)
# Output
# 42

M. Extended Iterable Unpacking

Extended iterable unpacking, introduced in Python 3, allows you to unpack an iterable into multiple variables, including capturing remaining items into another variable. It simplifies tasks such as splitting lists, assigning values from tuples, and processing variable-length iterables.

Unpacking a list into variables

my_list = [1, 2, 3, 4, 5]
first, *middle, last = my_list

print(first)
print(middle)
print(last)

# Output:
# 1
# [2, 3, 4]
# 5

Unpacking a string into variables

my_string = "Hello"
first, *rest = my_string

print(first)
print(rest)

# Output:
# H
# ['e', 'l', 'l', 'o']

Unpacking a tuple of unknown length

my_tuple = (1, 2, 3, 4, 5)
first, *middle, last = my_tuple

print(first)
print(middle)
print(last)

# Output:
# 1
# [2, 3, 4]
# 5

N. Multiple Inheritance and Method Resolution Order (MRO)

Python supports multiple inheritance, allowing a class to inherit from multiple base classes. Method Resolution Order (MRO) determines the order in which base classes are searched for a method. Understanding MRO helps you resolve method name conflicts and grasp the inheritance hierarchy.

class A:
    def say_hello(self):
        print("Hello from A")

class B(A):
    def say_hello(self):
        print("Hello from B")

class C(A):
    def say_hello(self):
        print("Hello from C")

class D(B, C):
    pass

d = D()
d.say_hello()

# Output:
# Hello from B

When we create an instance of D and call the say_hello() method, Python follows the Method Resolution Order (MRO) to determine which implementation of the method should be executed.

The MRO is determined by the C3 linearization algorithm, which is a consistent and predictable method resolution order. In this case, the MRO for class D would be [D, B, C, A, object]. Thus, say_hello() of class B gets invoked.

Blog