The Problem

We recently ran into a situation where some of our DynamoDB documents were reaching their max size limits.

Thanks to the flexibility offered by DynamoDB, teams had been adding keys without ever thinking about the document size (which is a generous one - 400Kb).

We also noticed that the documents in question had a lot of boolean attributes, We could have made a different partition and moved these attributes but that would mean changing a lot of query and getItem calls to batchGetItems.

The Solution
We instead took a different route and used the same document and infact added one more attribute to it - flags.
The bits of this attribute (of a number type) would represent all the boolean attributes we wanted to replace.

OfCourse, we had to store the order of flags as a separate configuration so we knew which bit meant which attribute but this could be loaded and cached in-memory as it was guaranteed to not change.

Here's what the records looked like after the migration:

// The fat DB item
{
  "flags": 24
  ...
}

// The configuration DB item
{
  "pk": "config#flagPosition",
  "value": ["key-1", "key-2", "key-3", ...]
}

To find out whether the flag was true or false all we needed was the position of the flag bit.

const isEnabled = (val, bitPosition) => {
    if (bitPosition > 30 || bitPosition < 0)
        throw new Error('Invalid flag position');
    return (val & (1 << bitPosition)) === 0 ? false : true;
};

isEnabled(24, config.indexOf('key-2')); // false
isEnabled(24, config.indexOf('key-3')); // true

To enable a flag:

const setEnabled = (val, bitPosition) => {
    if (bitPosition > 30 || bitPosition < 0)    
        throw new Error('Invalid flag position');
    return val | (1 << bitPosition);
};

let flags = 24;
flags = setEnabled(flags, config.indexOf('key-2'));
isEnabled(flags, config.indexOf('key-2')); // true

To disable a flag:

const setDisabled = (val, bitPosition) => {
    if (bitPosition > 30 || bitPosition < 0)
        throw new Error('Invalid flag position');
    const mask = ~(1 << bitPosition);
    return val & mask;
};
let flags = 26;
flags = setEnabled(flags, config.indexOf('key-2'));
isDisabled(flags, (flags, config.indexOf('key-2')); // true