Easy way to process waterfall functions

Have you ever encountered a process that flows like a waterfall? This is an example.

Even though we only hit 1 endpoint, we have to process all of them.

Yes, that's easy, but what if step 2 fails and we want everything to be repeated. But the data has already entered the database? Just delete it anyway, but it's lazy to handle it all.

Maybe the method I provide can make all of that easier, even if there are dozens of steps.

Disclaimer first, I implement this on nestjs. And I hope those of you who read already understand and are proficient in using nestjs.

We first prepare the base service, which contains the execute command to run all the steps in order.

import { Injectable } from '@nestjs/common';
import { FALLBACK_STEP_METADATA_KEY } from 'src/decorators/fallback.decorator';
import { STEP_METADATA_KEY } from 'src/decorators/step.decorator';
import { uid } from 'uid';

@Injectable()
export class WaterfallService {
  private steps: { methodName: string; order: number }[] = [];
  private fallbacks: { methodName: string; order: number }[] = [];

  constructor() {
    this.collectSteps();
  }

  private collectSteps() {
    const methods = Object.getOwnPropertyNames(Object.getPrototypeOf(this));
    methods.forEach((methodName) => {
      const order = Reflect.getMetadata(STEP_METADATA_KEY, this, methodName);
      if (order !== undefined) {
        this.steps.push({ methodName, order });
      }

      const fallbackOrder = Reflect.getMetadata(
        FALLBACK_STEP_METADATA_KEY,
        this,
        methodName,
      );
      if (fallbackOrder !== undefined) {
        this.fallbacks.push({ methodName, order: fallbackOrder });
      }
    });

    this.steps.sort((a, b) => a.order - b.order);
    this.fallbacks.sort((a, b) => a.order - b.order);
  }

  async executeSteps(params?) {
    const eventId = uid(6);
    let executedSteps = [];
    let returnedData: any;
    try {
      for (const step of this.steps) {
        let paramPassed = params;
        if (step.order > 1) {
          paramPassed = returnedData;
        }
        const result = await (this as any)[step.methodName](
          eventId,
          paramPassed,
        );
        returnedData = result;
        executedSteps.push(step);
      }
    } catch (error) {
      await this.executeFallbacks(executedSteps, eventId);
      throw error; // Re-throw the error after handling fallbacks
    }
  }

  private async executeFallbacks(executedSteps, eventId) {
    // Execute fallbacks in reverse order
    for (let i = executedSteps.length - 1; i >= 0; i--) {
      const step = executedSteps[i];
      const fallback = this.fallbacks.find((f) => f.order === step.order);
      if (fallback) {
        await (this as any)[fallback.methodName](eventId);
      }
    }
  }
}

Next, we will create a decorator to make it easier for the execute function to run the steps in the order we want.

import 'reflect-metadata';

export const STEP_METADATA_KEY = 'step_order';

export function Step(order: number): MethodDecorator {
  return (target, propertyKey, descriptor) => {
    Reflect.defineMetadata(STEP_METADATA_KEY, order, target, propertyKey);
  };
}

export const FALLBACK_STEP_METADATA_KEY = 'fallback_step_order';

export function Rollback(order: number): MethodDecorator {
  return (target, propertyKey, descriptor) => {
    Reflect.defineMetadata(FALLBACK_STEP_METADATA_KEY, order, target, propertyKey);
  };
}

The last one we implement into our service.

import { BadRequestException, Injectable } from '@nestjs/common';
import { WaterfallService } from './commons/waterfall/waterfall.service';
import { Step, Rollback } from './decorators/step.decorator';

@Injectable()
export class AppService extends WaterfallService {
  @Step(1)
  async logFirst(eventId) {
    console.log('Step 1 [eventId]:', eventId);
  }

  @Rollback(1)
  async fallbackFirst(eventId) {
    console.log('Rollback 1 [eventId]:', eventId);
  }

  @Step(2)
  async logSecond(eventId, data) {
    console.log('Step 2 [eventId]:', eventId);
  }

  @Rollback(2)
  async fallbackSecond(eventId) {
    console.log('Rollback 2 [eventId]:', eventId);
  }

  @Step(3)
  async logThird(eventId) {
    console.log('Step 3 [eventId]:', eventId);
  }

  @Rollback(3)
  async fallbackThird(eventId) {
    console.log('Rollback 3 [eventId]:', eventId);
  }

  async execute() {
    await this.executeSteps();
    return 'Step Executed';
  }
}

Now in the controller don't forget to call the execute function, like this example

import { Controller, Get } from '@nestjs/common';
import { AppService } from './app.service';

@Controller()
export class AppController {
  constructor(private readonly appService: AppService) {}

  @Get()
  getHello() {
    return this.appService.execute();
  }
}

Then run the nestjs. For the test hit [GET]http://localhost:3000

Later in the log terminal it will be like this

Now it's easy, right? You just add as many function steps as you need.

Now what if in step 3 there is an error and want to rollback the previous processes that have been run?
Take it easy, as long as there is a function with the decorator @Rollback(number)

The function will run if an error occurs. For example, here's an example.

import { BadRequestException, Injectable } from '@nestjs/common';
import { WaterfallService } from './commons/waterfall/waterfall.service';
import { Step, Rollback } from './decorators/step.decorator';

@Injectable()
export class AppService extends WaterfallService {
  @Step(1)
  async logFirst(eventId) {
    console.log('Step 1 [eventId]:', eventId);
  }

  @Rollback(1)
  async fallbackFirst(eventId) {
    console.log('Rollback 1 [eventId]:', eventId);
  }

  @Step(2)
  async logSecond(eventId, data) {
    console.log('Step 2 [eventId]:', eventId);
  }

  @Rollback(2)
  async fallbackSecond(eventId) {
    console.log('Rollback 2 [eventId]:', eventId);
  }

  @Step(3)
  async logThird(eventId) {
    throw new BadRequestException('Something error in step 3');
  }

  @Rollback(3)
  async fallbackThird(eventId) {
    console.log('Rollback 3 [eventId]:', eventId);
  }

  async execute() {
    await this.executeSteps();
    return 'Step Executed';
  }
}

Later, if it is run again, the results will be like this

I intentionally add eventId in each step, if there is a case you insert data into the database, also save the eventId to identify that the data has the eventId owner. So when you rollback and want to delete the data, you don't get confused about which data is being rolled back.

For the example case of data in the first function return that is passed to the next function, just return the function, then the return will be passed to the next function. This is the example

import { BadRequestException, Injectable } from '@nestjs/common';
import { WaterfallService } from './commons/waterfall/waterfall.service';
import { Step, Rollback } from './decorators/step.decorator';

@Injectable()
export class AppService extends WaterfallService {
  @Step(1)
  async logFirst(eventId, data) {
    console.log('Step 1 [eventId]:', eventId);
    console.log('Step 1 [data]:', data);

    return {
      step: 1,
      message: 'this data from step 1',
    };
  }

  @Rollback(1)
  async fallbackFirst(eventId) {
    console.log('Rollback 1 [eventId]:', eventId);
  }

  @Step(2)
  async logSecond(eventId, data) {
    console.log('Step 2 [eventId]:', eventId);
    console.log('Step 2 [data]:', data);

    return {
      step: 2,
      message: 'this data from step 2',
    };
  }

  @Rollback(2)
  async fallbackSecond(eventId) {
    console.log('Rollback 2 [eventId]:', eventId);
  }

  @Step(3)
  async logThird(eventId) {
    throw new BadRequestException('Something error in step 3');
  }

  @Rollback(3)
  async fallbackThird(eventId) {
    console.log('Rollback 3 [eventId]:', eventId);
  }

  async execute() {
    const data = { step: 0, message: 'this data from initial function' };
    await this.executeSteps(data);
    return 'Step Executed';
  }
}

Then the terminal will look like this

I added a repo example here.

That's all my method of running the waterfall function, Maybe if there are questions and collaboration, you can contact me.

Blog

Easy way to process waterfall functions

Alfian Rivaldi

Join Our Newsletter. No Spam, Only the good stuff.

Related