Towards Effortless Python Configuration Files Version 3

Introduction

This is the final article in this series. This implemenation seeks to fix the main disadvantage of boiler plate code that I described in the previous article. I call this implementation a dynamic property class.

Class Representation

The following class diagram shows the DynamicConfiguration reusable class and the supporting data structures needed for a developer to use this functionality. It still provides all the basic functionality of version 2 including auto-boot strapping, creation of missing sections and key values.

Developer Code Description

I am going to present the full source code for an application seeking to use this class. I am using the previous properties that we have been discussing in the previous 3 articles.

from codeallybasic.DynamicConfiguration import DynamicConfiguration
from codeallybasic.DynamicConfiguration import KeyName
from codeallybasic.DynamicConfiguration import SectionName
from codeallybasic.DynamicConfiguration import Sections
from codeallybasic.DynamicConfiguration import ValueDescription
from codeallybasic.DynamicConfiguration import ValueDescriptions
from codeallybasic.SecureConversions import SecureConversions

from codeallybasic.SingletonV3 import SingletonV3

from ByteSizedPython.ImpostorEnumByName import ImpostorEnumByName
from ByteSizedPython.PhoneyEnumByValue import PhoneyEnumByValue

LOGGER_NAME:       str = 'Tutorial'
BASE_FILE_NAME: str = 'config.ini'
MODULE_NAME:       str = 'version3properties'

DEFAULT_PHONEY_ENUM_BY_VALUE:  PhoneyEnumByValue  = PhoneyEnumByValue.FakeBrenda
DEFAULT_IMPOSTOR_ENUM_BY_NAME: ImpostorEnumByName = ImpostorEnumByName.High

GENERAL_PROPERTIES: ValueDescriptions = ValueDescriptions(
    {
        KeyName('debug'):    ValueDescription(defaultValue='False', deserializer=SecureConversions.secureBoolean),
        KeyName('logLevel'): ValueDescription(defaultValue='Info'),
        KeyName('phoneyEnumByValue'):  ValueDescription(defaultValue=DEFAULT_PHONEY_ENUM_BY_VALUE.value,  enumUseValue=True),
        KeyName('impostorEnumByName'): ValueDescription(defaultValue=DEFAULT_IMPOSTOR_ENUM_BY_NAME.name,  enumUseName=True),
    }
)

DATABASE_PROPERTIES: ValueDescriptions = ValueDescriptions(
    {
        KeyName('dbName'): ValueDescription(defaultValue='dbName'),
        KeyName('dbHost'): ValueDescription(defaultValue='localhost'),
        KeyName('dbPort'): ValueDescription(defaultValue='5342', deserializer=SecureConversions.secureInteger),
    }
)

CONFIGURATION_SECTIONS: Sections = Sections(
    {
        SectionName('General'):  GENERAL_PROPERTIES,
        SectionName('Database'): DATABASE_PROPERTIES,
    }
)

class ConfigurationPropertiesVersion3(DynamicConfiguration, metaclass=SingletonV3):
    def __init__(self):

        self._logger: Logger = getLogger(LOGGER_NAME)

        super().__init__(baseFileName=BASE_FILE_NAME, moduleName=MODULE_NAME, sections=CONFIGURATION_SECTIONS)

Lines 45-50 are the code that you have to write. Essentially, your are just ensuring that you pass the file name, module name, and the configuration sections. This Sections type comes from the DynamicConfiguration module.

Lines 21-28 and 30-36 are ValueDescriptions dictionaries. The KeyName is the property and points to a ValueDescription. Notice that the indicator on how to persist an enumeration is moved from the previous implementation's decorator to a boolean attribute in a ValueDescription.

Implementation Code Description

If you look closely at the class diagram for DynamicConfiguration you will see that it implements two Python magic methods. They are the __getattr__(self, name)__ and __setattr__(self, name, value)__ methods.

__getattr__(self, name)__ Allows a developer to define behavior when a class consumer attempts to access a non-existent attribute.
__setattr__(self, name, value)__ Allows a developer to define behavior for assignment to an attribute.

The following is the code for __getattr__. This looks very much like the decorator we used in version 2. The key work happens on the call on line 14 to the protected method _lookupKey(). It returns a full description of the attribute so that we can simulate the attribute retrieval.

    def __getattr__(self, attrName: str) -> Any:
        """
        Does the work of retrieving the named attribute from the configuration parser

        Args:
            attrName:

        Returns:  The correctly typed value
        """

        self._logger.info(f'{attrName}')

        configParser: ConfigParser     = self._configParser
        result:       LookupResult     = self._lookupKey(searchKeyName=KeyName(attrName))
        valueDescription: ValueDescription = result.keyDescription

        valueStr: str = configParser.get(result.sectionName, attrName)

        if valueDescription.deserializer is not None:
            value: Any = valueDescription.deserializer(valueStr)
        else:
            value = valueStr

        return value

The following is the__setattr__() implementation. Notice the support for enumerations in lines 22-27 and the write-through feature in line 30.

    def __setattr__(self, key: str, value: Any):
        """
        Do the work of writing this back to the configuration/settings/preferences file
        Ignores protected and private variables uses by this class

        Does a "write through" to the backing configuration file (.ini)

        Args:
            key:    The property name
            value:  Its new value
        """

        if key.startswith(PROTECTED_PROPERTY_INDICATOR) or key.startswith(PRIVATE_PROPERTY_INDICATOR):
            super(DynamicConfiguration, self).__setattr__(key, value)
        else:
            self._logger.debug(f'Writing `{key}` with `{value}` to configuration file')

            configParser: ConfigParser  = self._configParser
            result:       LookupResult  = self._lookupKey(searchKeyName=KeyName(key))
            valueDescription: ValueDescription = result.keyDescription

            if valueDescription.enumUseValue is True:
                valueStr: str = value.value
                configParser.set(result.sectionName, key, valueStr)
            elif valueDescription.enumUseName is True:
                configParser.set(result.sectionName, key, value.name)
            else:
                configParser.set(result.sectionName, key, str(value))

            self.saveConfiguration()

Accessing and Modifying Properties

Accessing and modifying properties is exactly the same as in version 2.

    basicConfig(level=INFO)

    config: ConfigurationPropertiesVersion2 = ConfigurationPropertiesVersion2()

    logger: Logger = getLogger(LOGGER_NAME)

    logger.info(f'{config.debug=}')
    logger.info(f'{config.logLevel=}')
    logger.info(f'{config.phoneyEnumByValue=}')
    logger.info(f'{config.impostorEnumByName=}')
    logger.info('Database Properties Follow')
    logger.info(f'{config.dbName=}')
    logger.info(f'{config.dbHost=}')
    logger.info(f'{config.dbPort=}')
    logger.info('Mutate Enumeration Properties')
    config.phoneyEnumByValue = PhoneyEnumByValue.TheWanderer
    logger.info(f'{config.phoneyEnumByValue=}')
    config.impostorEnumByName = ImpostorEnumByName.Low
    logger.info(f'{config.impostorEnumByName=}')

The above snippet produces the following output.

INFO:Tutorial:config.debug='False'
INFO:Tutorial:config.logLevel='Info'
INFO:Tutorial:config.phoneyEnumByValue=<PhoneyEnumByValue.FakeBrenda: 'Faker Extraordinaire'>
INFO:Tutorial:config.impostorEnumByName='High'
INFO:Tutorial:Database Properties Follow
INFO:Tutorial:config.dbName='example_db'
INFO:Tutorial:config.dbHost='localhost'
INFO:Tutorial:config.dbPort=5432
INFO:Tutorial:Mutate Enumeration Properties
INFO:Tutorial:config.phoneyEnumByValue=<PhoneyEnumByValue.TheWanderer: 'The Wanderer'>
INFO:Tutorial:config.impostorEnumByName='Low'

Conclusion

The source code for this article is here. See the support class SingletonV3. See the implementation of

Advantages

Easy type safe access to application properties
Reusable parent class for different implementations
Data structure driven code to add new sections and configuration keys
No boiler plate code for properties

Disadvantages

Since there are no actual properties implemented we do not get IDE support for them
Additionally, because of the lookup methods for keys, different keys in different sections cannot have the same name

Blog