Integrating Argo CD and Slack for Real-time Notifications

In the dynamic landscape of Kubernetes application deployment, GitOps has emerged as a transformative methodology. At its core lies the concept of declarative configuration stored in version-controlled repositories, enabling consistent and automated application management. Argo CD, a leading tool in the GitOps ecosystem, empowers organizations to efficiently deploy and manage Kubernetes applications with precision and reproducibility.

As GitOps gains traction, the need for real-time communication and collaboration surrounding deployment updates becomes increasingly apparent.

In this blog post, we will explain Argo CD Notifications, go through the process of creating a Slack channel and Slack app, set up Argo CD with notifications, integrate Slack, and see notifications in action for an improved DevOps workflow. Let’s kick off our exploration by gaining a deep understanding of the inner workings of Argo CD Notifications.

Argo CD Notifications

Delivering the right version of an application to users is not just a matter of functionality; it’s essential for user satisfaction and system stability.

Imagine an application is released with the wrong tag and your users are not able to access it, leading to dissatisfied customers. How can we avoid it? We can employ various strategies, such as rigorous testing, automation, notifications, and standardized release processes, to prevent such mishaps.

Now, picture this scenario: a notification is sent out about the degraded state of the application, enabling immediate action to revert to the previous version until the correct one is ready. This highlights the vital role of notifications in promptly detecting and rectifying deployment errors.

This is where Argo CD Notifications steps in, bridging the gap between the technical intricacies of GitOps and the need for timely, actionable insights. Argo CD Notifications provides a suite of triggers that can be configured to notify users about various events in the deployment process. These events include application creation, deletion, deployment, sync failures, health degradation, and more.

Argo CD Notifications doesn’t restrict you to a single communication method. Beyond Slack integration, it offers a variety of notification options tailored to your preferences. These include sending notifications through email for direct inbox visibility, utilizing webhooks to connect with other communication platforms and services, and even allowing customization for integration with your existing notifications tools and systems.

With these various notification services at your disposal, Argo CD Notifications ensures that you’re promptly informed of any changes or issues, preventing disjointed efforts, reducing response times, and minimizing potential downtimes.

How does Argo CD Notifications work?

Argo CD Notifications controllers employ event listeners to capture events originating from the Argo CD application controller, which manages multiple applications. The ArgoCD application controller actively monitors the repository for any modifications, triggering events when changes are detected. These events are then captured by the event listener of Argo CD Notifications controller, which subsequently forwards them to various Argo CD Notifications services. These notification services are responsible for dispatching notifications to designated platforms or channels, such as Slack, using webhooks and other methods as required. The diagram below offers a detailed overview of how notifications are delivered to a Slack channel through the Argo CD notifications controller and Slack notifications service.

Let’s understand the function of each component from the above diagram:

Argo CD Server: This is the Argo CD control plane responsible for managing applications and their deployments.

Notifications Controller: This component is in charge of monitoring Argo CD events. It can detect changes in applications or deployments and trigger notifications when specific events occur.

Event Listener (Webhook, etc.): This could be a webhook, custom application, or event listener that interacts with the Argo CD API to capture events like application deployments or sync status changes.

Slack Notifications Service: A Slack Notifications service integrates with an application’s Event Listener to send event data to designated Slack channels or users, allowing real-time event-based notifications in Slack.

Slack Channel (Notifications): The Slack Notifications service posts the formatted event data to a dedicated Slack channel where team members can see notifications and respond as needed.

Prerequisites

Kubernetes Cluster
Kubectl client
A Slack Account and Workspace
Argo CD
Demo application - We will be using a Guestbook application tweaked for our use case. (Original source argocd-example-apps)

Now, let’s delve into the practical process of setting up Slack integration, which forms a pivotal part of the Argo CD Slack Notifications service.

Getting started with Slack integration

Now that you have the necessary prerequisites in place, let’s begin by setting up a dedicated Slack channel for sending notifications. If you already have one then you can skip this step.

Create a dedicated Slack channel
To streamline our notifications, we’ve created a private and dedicated Slack channel named argocdnotification. If you require assistance in creating a channel, please refer to Slack’s official guide for guidance.

Create a Slack app for notifications

1.Creating a Slack app will ensure secure and authorized communication between two platforms - Argo CD and Slack, enabling Argo CD to send notifications to Slack channels. To initiate the Slack app creation process, you can begin by browsing this URL. Once the page opens, click on ‘Create New App’ option.

2.Select From scratch option

3.Name your app, select your workspace, and then create the application.

4.Provide the required permissions to the application - Navigate to the OAuth & Permissions section on the left-hand panel. Within the Scope section, include both chat:write and chat:write.customize permissions. The chat:write permission allows the Slack app to send messages to channels and users, while chat:write.customize specifically permits the customization of messages, including formatting and attachments, enhancing the flexibility of communication within Slack.

5.Once you’ve configured the scope, proceed to install the app in your workspace.

6.Allow the requested permissions.

7.Copy the Bot User OAuth Token and keep it saved for later use in the Argo CD Notifications configurations.

8.Now let’s integrate the Slack application into the Argo CD Notifications Slack channel. To include the app in the Slack channel, go to the Slack channel ‘argocdnotification’ (that we had created prior). In the chat window, type the app name, @Argo CD Notifications, and then paste it (Bot User OAuth Token). You’ll receive an invitation option to add the app to the channel. Click ‘Invite to Channel’.

We’ve successfully created a dedicated Slack channel and added the notifications app. Now, let’s proceed with the installation of Argo CD Notifications triggers and templates.

Argo CD Notifications triggers and templates installation

Now we will set up Argo CD Notifications, which provides a collection of pre-configured triggers and templates, eliminating the need to create them from scratch. To do this, apply the following command to your Argo CD installation in the “argocd” namespace:

kubectl apply -n <argocd-namespace> -f \
https://raw.githubusercontent.com/argoproj-labs/argocd-notifications/release-1.0/catalog/install.yaml

After applying this configuration, you can validate the triggers and templates that have been added to the “argocd-notifications-cm” ConfigMap within the “argocd-namespace using the following command:

kubectl get cm argocd-notifications-cm -n <argocd-namespace> -o yaml

When installing Argo CD using a Helm chart, you can add default triggers and template by copying the sections notifications.templates and notifications.triggers from the default values.yaml file. If you are installing Argo CD with Argo CD operator then follow the Argo CD operator notifications installation docs for detailed steps.

Configure secrets and tokens

We have Argo CD and Slack setup ready, it’s time to integrate both. For the same, we will update the Argo CD Notifications secret to include your Slack app token and make adjustments to the argocd-notifications-cm ConfigMap to enable Slack integration. The Argo CD Notifications ConfigMap centralizes and manages notifications settings, facilitating easy customization and updates of deployment event notifications within ArgoCD.

Now let’s start with configuring the Argo CD Notifications secret.

For manifest-based or operator-based installation
1.Let’s add your Slack app OAuth token to argocd-notifications-secretl Secret: Replace the placeholder “xoxb-xxxx” with your actual Slack token value before executing the following command:

 kubectl patch secret argocd-notifications-secret -n <argocd-namespace> \
 --type merge --patch '{"stringData":{"slack-token": "xoxb-xxxx"}}'

To check if the Slack token has been successfully added to the Secret, run:

 kubectl get secret argocd-notifications-secret -n <argocd-namespace> -o yaml

The Slack token will be encoded in the Secret, ensuring its security:

 apiVersion: v1
 kind: Secret
 metadata:
   labels:
     app.kubernetes.io/component: notifications-controller
     app.kubernetes.io/name: argocd-notifications-controller
     app.kubernetes.io/part-of: argocd
   name: argocd-notifications-secret
   namespace: <argocd-namespace>
 data:
   slack-token: eG94Yi14eHh4
 type: Opaque

2.Configure Slack integration in the argocd-notifications-cm ConfigMap:

Let’s add Slack Notifications Service along with reference to your Slack token in argocd-notifications-cm ConfigMap with below command:

  kubectl patch configmap argocd-notifications-cm -n <argocd-namespace> --type merge -p '{"data": {"service.slack": "token: $slack-token"}}'

For Helm-Based installation
If you’re using a Helm chart to install Argo CD, you can achieve the same by adding Slack app OAuth token in notifications.secret.items section and reference to the Slack token in notifications.notifiers within your values.yaml file:

notifications:
  secret:
    create: true
    items: 
      slack-token: xoxb-xxxx
  notifiers: 
    service.slack: |
      token: $slack-token

With these configurations, we’ve successfully updated your secrets, tokens, and integrated Slack with Argo CD Notifications for a more effective notification system.
Now we’ve got our notification system all set up, it’s time to test if it’s working correctly or not. Let’s deploy a demo application and fine-tune the notification settings.

Configuring Argo CD application for notifications

To initiate the demo Argo CD application, we’ll utilize the guestbook application from our argocd-notifications repository. Let’s start with creating the demo application in Argo CD.

To create the demo application, follow these steps:

1.Log in to Argo CD through the CLI based on how you’ve exposed it.

2.Now, proceed to launch the demo application:

 argocd app create guestbook --repo https://github.com/DadaGore/argocd.git \
  --path guestbook --dest-namespace guestbook \
  --dest-server https://kubernetes.default.svc --directory-recurse \
  --annotations notifications.argoproj.io/subscribe.on-sync-succeeded.slack=argocdnotification

3.We have added the annotation on the application to subscribe to Argo CD events. Let’s understand the format of the subscription. The subscription to Argo CD application events can be defined using below annotation format.

 notifications.argoproj.io/subscribe.<trigger>.<service>: <recipient>

Now, let’s break down the annotation components for your application:

on-sync-succeeded (Trigger Name): This component specifies the event trigger. In this case, it’s set to “on-sync-succeeded,” meaning the annotation is triggered when a synchronization succeeds. You can find comprehensive information on triggers in the official Argo CD documentation.
slack (Notification Service Name): In this context, ‘Slack’ designates the specific notification service or destination where you intend to deliver notifications. Argo CD offers various notification service options, including alertmanager, email, github, and more, which can be explored in detail within the Argo CD documentation.
argocdnotification (List of Recipients): This part contains a list of recipients who will receive the notifications. Recipients are separated by semicolons (;). In your case, it’s the Slack channel ‘argocdnotification’.

4.Finally, synchronize the application by executing this command:

 argocd app sync guestbook

Following a successful application synchronization, a notification will be automatically sent to the Slack channel “argocdnotification” as we’ve subscribed to on-sync-succeeded event with relevant annotation in the application configuration.

Our application deployment has been a success. Initially, we incorporated annotations to specifically capture sync-succeeded events. Now, to understand how Argo CD can help us to notify in case of any issues - let’s take the next step and launch an additional instance of the Guestbook application from the “guestbook_degraded” directory. In this instance, we have intentionally introduced health degradation by using an incorrect image tag. This action will enable us to receive alerts when the health of the application deteriorates. To achieve this, we’ve extended the annotations in the application manifest to include monitoring for various other events. You can execute the following command to launch the application and apply these annotations to the ArgoCD application.

# Launch guestbook1 application from guestbook-degraded directory
 argocd app create guestbook1 --repo https://github.com/infracloudio/argocd-notifications.git\
  --path guestbook-degraded --dest-namespace guestbook1 \
  --dest-server https://kubernetes.default.svc --directory-recurse \
  --annotations notifications.argoproj.io/subscribe.on-sync-succeeded.slack=argocdnotification
 # Add annotations to the application
 argocd app patch guestbook1 --patch '[{"op": "add", "path": "/metadata/annotations/notifications.argoproj.io~1subscribe.on-sync-failed.slack", "value": "argocdnotification"}, 
  {"op": "add", "path": "/metadata/annotations/notifications.argoproj.io~1subscribe.on-sync-status-unknown.slack", "value": "argocdnotification"},
  {"op": "add", "path": "/metadata/annotations/notifications.argoproj.io~1subscribe.on-deployed.slack", "value": "argocdnotification"}, 
  {"op": "add", "path": "/metadata/annotations/notifications.argoproj.io~1subscribe.on-deleted.slack", "value": "argocdnotification"}, 
  {"op": "add", "path": "/metadata/annotations/notifications.argoproj.io~1subscribe.on-health-degraded.slack", "value": "argocdnotification"}]'

Upon executing the above command, you will observe the following annotations applied to the application:

metadata:
 annotations:
     # To send Slack notifications when a resource is deleted in Argo CD.
     notifications.argoproj.io/subscribe.on-deleted.slack: argocdnotification
     # To send Slack notifications when a deployment is completed in Argo CD.
     notifications.argoproj.io/subscribe.on-deployed.slack: argocdnotification
     # To send Slack notifications when the health of a resource degrades in Argo CD.
     notifications.argoproj.io/subscribe.on-health-degraded.slack: argocdnotification
     # To send Slack notifications when a synchronization fails in Argo CD.
     notifications.argoproj.io/subscribe.on-sync-failed.slack: argocdnotification
     # To send Slack notifications when the synchronization status becomes unknown in Argo CD.
     notifications.argoproj.io/subscribe.on-sync-status-unknown.slack: argocdnotification
     # To send Slack notifications when a synchronization succeeds in Argo CD.
     notifications.argoproj.io/subscribe.on-sync-succeeded.slack: argocdnotification

Now, as soon as the application’s status transitions to a degraded state, you will receive notifications to keep you informed. This enhancement in monitoring ensures that you stay ahead of potential issues and can take proactive measures to maintain the health of your application.

It’s worth noting that the Argo CD Notifications system seamlessly bridges the gap between your Kubernetes deployments and real-time communication. With timely notifications sent to the dedicated Slack channel “argocdnotification,” your SRE as well as DevOps teams stay informed and responsive to deployment events or any failures.

Conclusion

In this blog post, we saw how you can streamline the integration of Argo CD Notifications with Slack, a crucial step in optimizing Kubernetes application deployment and management. GitOps, with its declarative and version-controlled approach, is at the forefront of modern application management, and Argo CD Notifications enhances it further.

By creating a dedicated Slack channel, installing Argo CD Notifications, and configuring your Argo CD application, you have established a powerful system for real-time communication and collaboration. This integration ensures timely updates, quick issue resolution, and efficient Kubernetes application management, empowering you to navigate the Kubernetes landscape with precision and confidence.

I hope you found this blog post informative and engaging. For more posts like this one, do subscribe to our newsletter for a weekly dose of cloud native. I’d love to hear your thoughts on this post, let’s connect and start a conversation on LinkedIn.

Looking for help with GitOps adoption using Argo CD? Do check our Argo CD consulting capabilities and expertise to know how we can help with your GitOps adoption journey. If you’re looking for managed on-demand Argo CD support, check our support model.

References

Getting Started With Argo CD
Argo CD notifications

Community Support

If you want to connect to the Argo CD community, please join CNCF Slack. You can join #argo-cd and many other channels too.

Blog