Collect what you need, only when you need it

Sidekick is an open source live application debugger that comes with many built-in features to make data collection both easier and more efficient. Collecting snapshots and generating logs from running applications are powerful for learning how your code works as long as you collect the data you need.

Logging everything is not a smart move as you can find yourself searching in a log pile full of mostly unnecessary logs in a very short time. Similar case applies for the snapshots & traces. That is why we have included conditional data generation in Sidekick. Combined with hit and time limits, they allow you to collect only what you need, leaving a graveyard of useless data behind.

In addition to our out-of-the-box conditional limitation features, Sidekick’s API’s and clients let you to bring your own rules which leads to a relatively new approach to observability, a rule based one. With this approach we will be able to put tracepoints when rules met and start collecting data automatically based on rule engine’s output.

Rule Engines

There are plenty of different rule engines & workflow automation tools that we can use for such a scenario. A few can be listed as:

The Universal Process Orchestrator | Camunda

Camunda's process automation platform allows developers to design, automate and improve processes. Start your free trial today.

camunda.com

Drools - Drools - Business Rules Management System (Java™, Open Source)

Drools introduces the Business Logic integration Platform which provides a unified and integrated platform for Rules, Workflow and Event Processing.

drools.org

json-rules-engine - npm

Rules Engine expressed in simple json. Latest version: 6.1.2, last published: a year ago. Start using json-rules-engine in your project by running `npm i json-rules-engine`. There are 102 other projects in the npm registry using json-rules-engine.

npmjs.com

node-rules - npm

Business Rules Engine for JavaScript. Latest version: 7.2.0, last published: 24 days ago. Start using node-rules in your project by running `npm i node-rules`. There are 15 other projects in the npm registry using node-rules.

npmjs.com

We will be using Camunda for this example for broader use cases as Camunda is more than a rule engine it is a universal process orchestrator. For a usage example of Camunda you can check out:

Use Camunda as an easy-to-use REST-based orchestration and workflow engine (without touching Java)

Using Sidekick with Camunda

Camunda provides external task clients that allow developers to implement tasks to run when a workflow hits to the related step. We will be using Node.js client but cases below can also be achieved with different clients of Camunda and Sidekick’s upcoming Python & Java clients.

camunda-external-task-client-js - npm

Implement your [BPMN Service Task](https://docs.camunda.org/manual/latest/user-guide/process-engine/external-tasks/) in NodeJS.. Latest version: 2.2.0, last published: 5 months ago. Start using camunda-external-task-client-js in your project by running `npm i camunda-external-task-client-js`. There are 15 other projects in the npm registry using camunda-external-task-client-js.

npmjs.com

Example task method:

// Subscribe to the topic: 'topicName'  
client.subscribe("topicName", async function({ task, taskService }) {  
  // Put your business logic  
  // Complete the task  
  await taskService.complete(task);  
});

@runsidekick/sidekick-client - npm

Sidekick Node.js Client. Latest version: 0.0.2, last published: a month ago. Start using @runsidekick/sidekick-client in your project by running `npm i @runsidekick/sidekick-client`. There are no other projects in the npm registry using @runsidekick/sidekick-client.

npmjs.com

Task method with Sidekick Put tracepoint event:

// Subscribe to the topic: 'topicName'  
client.subscribe("topicName", async function({ task, taskService }) {    const params= {  
      applicationFilters: [  
          {  
            name: "Demo application",  
            version: "v1.0",  
            stage: "prod"  
          }  
        ],  
      fileName: "gitlab.com/repos/...",  
      lineNo: 23,  
      expireSecs: -1,  
      expireCount: -1,  
      enableTracing: true,  
      persist: true  
  }  
  apiClient.putTracepoints(params);  
  await taskService.complete(task);  
});

Example use cases

Case 1 — Activating Tracepoints based on error scenarios

Camunda’s DMN engine provides us ability to define rules in an excel like interface.

Sidekick DMN

As you can observe at the DMN above we can define where we put tracepoints when a certain type of error occurs.

Triggering workflows with error types makes a great use case as it enables us to automatically start collecting extra information based on different cases. Enabling different tracepoints at once is especially useful for spotting state changes.

Note: In addition to this use case Sidekick Node.js agent has ability to automatically capture error stacks, without any tracepoint. check out: https://www.npmjs.com/package/@runsidekick/sidekick-agent-nodejs

Example Sidekick BPMN

Case 2 —Putting Tracepoints on workflow errors

image from: https://blog.bernd-ruecker.com/use-camunda-without-touching-java-and-get-an-easy-to-use-rest-based-orchestration-and-workflow-7bdf25ac198e

Camunda comes with an error handling mechanism that lets you run actions in cases of errors. In addition to Sidekick client, we can also attach Sidekick agents to task services to observe their state changes and debug them effectively.

Case 3 — Activating Tracepoints on certain events

Last but not least, Sidekick Actions bring more than debugging to the table. They are also great tools to understand how the system works. Especially for the cases like understanding why and how edge cases occur or understanding when does a certain step is reached.

We have briefly explained how Sidekick can be used with Camunda to achieve rule based observability, for better understanding systems, having a smoother debug experience and lowering monitoring costs with conditional data collection. The 3 cases we have explained are just the tip of the iceberg and there is a lot more to discover here.

Please let us know your invaluable ideas because they will guide us on our path to discovering the future of application observability. You can get in touch with us through Twitter, and join our Discord community.