Automating CI/CD Pipelines for Kubernetes with Argo Rollouts, Argo CD, Argo Workflow & Events

As DevOps continues to evolve, GitOps has emerged as a key practice for managing Kubernetes deployments. By treating your Git repository as the single source of truth, you can ensure reliable, repeatable, and automated deployments. This blog will walk you through a complete infrastructure setup and deployment pipeline for a Node.js API using Argo Rollouts, Argo CD, and Argo Events.

Find the complete code and configuration for this setup on GitHub:
GitHub Repository Link Infra

GitHub Repository Link Node API Example

Why Argo Rollouts?

Traditional Kubernetes deployments are robust, but they lack built-in support for progressive delivery strategies. Argo Rollouts fills this gap by providing:

Blue-Green Deployments: Seamlessly shift traffic between environments.
Canary Rollouts: Incrementally release features to minimize risk.
Real-Time Monitoring: Observe application health during rollouts.

By incorporating Argo Rollouts into our workflow, we can enhance the reliability of Kubernetes-based applications while maintaining GitOps principles.

Part 1: Infrastructure & Rollout Strategy

Installing Kind

Kind is essential for creating local Kubernetes clusters for testing. Install it using:

brew install kind

Installing Argo Rollouts CLI

The Argo Rollouts plugin manages blue-green and canary deployments. Install it based on your OS:

macOS:

curl -LO https://github.com/argoproj/argo-rollouts/releases/latest/download/kubectl-argo-rollouts-darwin-arm64
chmod +x ./kubectl-argo-rollouts-darwin-arm64
sudo mv ./kubectl-argo-rollouts-darwin-arm64 /usr/local/bin/kubectl-argo-rollouts

Linux:

curl -LO https://github.com/argoproj/argo-rollouts/releases/latest/download/kubectl-argo-rollouts-linux-amd64
chmod +x ./kubectl-argo-rollouts-linux-amd64
sudo mv ./kubectl-argo-rollouts-linux-amd64 /usr/local/bin/kubectl-argo-rollouts

Creating Multi-Node Clusters

Set the Kubernetes configuration file:
```
export KUBECONFIG=$PWD/kubeconfig.yaml
```

Create a cluster using Kind:

kind create cluster --name argo-cluster --config kind-config.yaml

Verify the cluster nodes:
```
kubectl get nodes
```

Setting Up NGINX Ingress Controller

Deploy an ingress controller for managing HTTP traffic:

helm upgrade --install ingress-nginx ingress-nginx \
  --repo https://kubernetes.github.io/ingress-nginx \
  --namespace ingress-nginx --create-namespace \
  --set controller.hostPort.enabled=true \
  --set controller.service.type=NodePort \
  --set controller.hostPort.ports.http=80 \
  --set-string controller.nodeSelector."kubernetes\.io/os"=linux \
  --set-string controller.nodeSelector.ingress-ready="true"

Label the worker node:

kubectl label node argo-cluster-worker ingress-ready=true

Verify the ingress controller:

kubectl get pods -n ingress-nginx

Setting Up Argo Rollouts

Create a namespace for Argo Rollouts:
```
kubectl create namespace argo-rollouts
```

Download and apply the Argo Rollouts controller:

curl -L -o argo-controller.yaml https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml
kubectl apply -n argo-rollouts -f argo-controller.yaml

Verify the installation:

kubectl api-resources | grep -i argo
kubectl argo rollouts version

Docker Secrets for Kubernetes

Generate Docker Hub access tokens for private image pulls:

Create Docker secrets for the staging namespace:

kubectl create secret -n staging docker-registry docker-registry-creds \
   --docker-server=https://index.docker.io/v1/ \
   --docker-username=<username> --docker-password=<password>

Repeat for the production namespace.

```bash
kubectl create secret -n prod docker-registry docker-registry-creds \
   --docker-server=https://index.docker.io/v1/ \
   --docker-username=<username> --docker-password=<password>
```

Deploying Staging with Blue-Green Strategy

Change directory to the application’s kustomize folder:
```
cd apps/node-demo/
```
Apply the deployment:
```
kubectl apply -k staging
```

Verify the deployment:

kubectl -n staging get all
kubectl -n staging get ingress

Update /etc/hosts:

echo "127.0.0.1 staging.node-api.example.com" | sudo tee -a /etc/hosts

Test the API:
- Swagger UI: http://staging.node-api.example.com/swagger/
- Preview: http://staging.node-api.example.com/swagger/preview

Watch rollouts:

kubectl argo rollouts dashboard -p 3100

open a new terminal (terminal 2) and run this command:

kind export kubeconfig --name argo-cluster

Deploying Production with Canary Strategy

Change directory:
```
cd apps/node-demo/
```
Deploy the application to production:
```
kubectl apply -k prod
```

Verify production deployment:

kubectl -n prod get all
kubectl -n prod get ingress

Update /etc/hosts:

echo "127.0.0.1 prod.node-api.example.com" | sudo tee -a /etc/hosts

Access the production API:

- `http://prod.node-api.example.com/swagger`
- `http://prod.node-api.example.com/swagger/preview`

Test the Rollout with a Patch:

TIMESTAMP=$(date +'%Y%m%d%H%M%S')
kubectl -n staging patch rollout node-api --type='json' -p="[{
    \"op\": \"add\",
    \"path\": \"/spec/template/metadata/annotations\",
    \"value\": {\"force-update\": \"$TIMESTAMP\"}
}]"

Setting Up Continuous Deployment with Argo CD

In this part we’ll focus on integrating Argo CD, a powerful continuous deployment tool that allows you to automate and sync your Kubernetes applications directly from Git repositories. This step ensures your deployment pipelines are seamless, automated, and fully aligned with GitOps principles.

Why Argo CD?

Argo CD enhances your deployment process by providing:

GitOps Automation: Automatically sync your Kubernetes clusters with your Git repository.
Declarative Configurations: Manage everything via manifests stored in Git.
Rollbacks: Easily roll back to previous application states.
Visibility: Monitor deployment health through a user-friendly UI and CLI.

This makes Argo CD an ideal choice for handling Kubernetes workloads at scale.

Step 1: Install Argo CD

to avoid any confusion, we're still in terminal 2, so cd back to the root of the folder

To start, install Argo CD on your Kubernetes cluster.

Create a namespace for Argo CD:
```
kubectl create namespace argocd
```

Apply the Argo CD manifest:

kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

Verify the Argo CD components:
```
kubectl get all -n argocd
```

Wait until all the components are in the Running state.

Step 2: Configure Password & Server Access

Next, secure the Argo CD server with a custom admin password.

Generate a secure password and hash:

PASSWORD=$(openssl rand -base64 16)
BCRYPT_HASH=$(htpasswd -bnBC 10 "" "$PASSWORD" | tr -d ':\n')

Set the password in the Argo CD secret:

kubectl -n argocd patch secret argocd-secret \
  -p '{"stringData": {
    "admin.password": "'"$BCRYPT_HASH"'",
    "admin.passwordMtime": "'$(date +%FT%T%Z)'"
  }}'

Restart the Argo CD server to apply changes:

kubectl -n argocd rollout restart deployment argocd-server

Expose the server using NodePort:

kubectl patch svc argocd-server -n argocd --patch '{"spec": {"type": "NodePort", "ports": [{"nodePort": 32100, "port": 443, "protocol": "TCP", "targetPort": 8080}]}}'

Access the Argo CD UI:

- Forward the port:

    kubectl port-forward svc/argocd-server 8080:80 -n argocd

- Open in your browser: `http://localhost:8080`

in terminal 3 run this command

kind export kubeconfig --name argo-cluster

Step 3: Install and Configure the Argo CD CLI

To interact with Argo CD via the CLI:

Download the CLI:

sudo curl -sSL -o /usr/local/bin/argocd https://github.com/argoproj/argo-cd/releases/latest/download/argocd-darwin-amd64

Make it executable:
```
sudo chmod +x /usr/local/bin/argocd
```
Login to Argo CD:
```
argocd login localhost:8080
```
Verify the CLI version:
```
argocd version
```

Step 4: Integrate Git with Argo CD

To sync repositories with Argo CD, configure SSH keys for secure access.

Step 4.1: Create a New SSH Key

Generate an SSH key pair:

ssh-keygen -t ed25519 -C "your_email@example.com" -f ~/.ssh/new_github_key

- Replace `your_email@example.com` with your GitHub email.

Add the public key to GitHub:

- Copy the public key:

    ```bash
    cat ~/.ssh/new_github_key.pub
    ```

- Go to **GitHub > Settings > SSH and GPG keys > New SSH key**, and paste the key.

Step 4.2: Configure the SSH Key for GitHub

Edit your SSH config:

nano ~/.ssh/config

Add:

Host github-new
    HostName github.com
    User git
    IdentityFile ~/.ssh/new_github_key

Test the SSH connection:

ssh -T github-new

You should see:

Hi <username>! You've successfully authenticated, but GitHub does not provide shell access.

Step 4.3: Add the Repository to Argo CD

Add the repository:

argocd repo add git@github.com:your-username/your-repo.git \
  --ssh-private-key-path ~/.ssh/new_github_key \
  --name argo-boot

Verify the repository:
```
argocd repo list
```

Step 5: Deploy Applications Using Argo CD

Ensure the Github repos are created and create a release branch as it is needed.

Navigate to your Argo CD project:
```
cd infra/argo-cd/projects/
```
Apply the project YAML:
```
kubectl apply -f node-demo.yaml
```

Deploy staging and production applications:

cd ../applications
kubectl apply -f node-demo-staging.yaml
kubectl apply -f node-demo-prod.yaml

Verify deployments in the Argo CD UI:

- Open: `http://localhost:8080/applications`

Setting Up Continuous Integration with Argo Workflows

In this part we’ll focus on building a Continuous Integration (CI) pipeline using Argo Workflows and Argo Events. These tools form the backbone of an automated system that responds to changes in your Git repositories, builds your application, and prepares it for deployment.

Why Argo Workflows and Argo Events?

Argo Workflows: A Kubernetes-native workflow engine designed for large-scale automation. Perfect for tasks like building Docker images or running tests.
Argo Events: A Kubernetes event-based dependency manager that triggers workflows based on events like Git pushes or webhooks.

Together, these tools automate CI pipelines with real-time event-driven capabilities.

cd back to the root folder

Step 1: Install Argo Workflows

1. Create a namespace for Argo Workflows:

kubectl create namespace argo

2. Apply the Argo Workflows manifest:

ARGO_WORKFLOWS_VERSION="v3.6.0"
kubectl apply -n argo -f "https://github.com/argoproj/argo-workflows/releases/download/${ARGO_WORKFLOWS_VERSION}/quick-start-minimal.yaml"

3. Verify the installation:

kubectl get all -n argo

Wait until all components are in the Running state.

4. Port-forward to access the Argo Workflows UI:

kubectl -n argo port-forward service/argo-server 2746:2746

Access the Argo Workflows UI in your browser:

https://localhost:2746

now in the final terminal, terminal 4, run this command

kind export kubeconfig --name argo-cluster

Step 2: Install and Configure the Argo CLI

Set your system architecture:

ARGO_OS="darwin"  # macOS users
ARGO_ARCH="amd64" # Default architecture

# Adjust for M1 (ARM-based architecture):
if [[ $(uname -m) == "arm64" ]]; then
  ARGO_ARCH="arm64"
fi

Download the Argo CLI:

curl -sLO "https://github.com/argoproj/argo-workflows/releases/download/v3.6.0/argo-$ARGO_OS-$ARGO_ARCH.gz"

Unzip the binary:
```
gunzip "argo-$ARGO_OS-$ARGO_ARCH.gz"
```
Make the binary executable:
```
chmod +x "argo-$ARGO_OS-$ARGO_ARCH"
```

Move the binary to your system path:

sudo mv "./argo-$ARGO_OS-$ARGO_ARCH" /usr/local/bin/argo

Verify the installation:
```
argo version
```

Step 3: Configure GitHub Integration

Generate a Personal Access Token (PAT)

Go to GitHub Personal Access Tokens.
Click Generate new token and select:
- repo for private repository access.
- Optionally, read:org for organizational repositories.
Copy the token once it’s generated.

Create Kubernetes Secrets for GitHub Integration

if you run into any issues, make sure all 4 secrets are created

Create a namespace for Argo Events:
```
kubectl create namespace argo-events
```

Add your GitHub credentials as a secret:

kubectl create secret generic git-credentials \
  --from-literal=username=<your-username> \
  --from-literal=token=<your-github-token> \
  -n argo

For events, create a GitHub token secret:

kubectl create secret generic github-token-secret \
  --from-literal=token=<your-github-token> \
  --namespace=argo-events

Add Docker credentials:

kubectl create secret -n argo docker-registry docker-registry-creds \
  --docker-server=https://index.docker.io/v1/ \
  --docker-username=<docker-username> \
  --docker-password=<docker-password>

Add Github credentials for argo-events (different than above)

kubectl create secret generic git-credentials \
  --from-literal=username=mtyiska \
  --from-literal=token=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  -n argo-events

Step 4: Set Up Argo Events

Install Argo Events

Apply the Argo Events manifests:

kubectl apply -f https://raw.githubusercontent.com/argoproj/argo-events/stable/manifests/install.yaml 
kubectl apply -f https://raw.githubusercontent.com/argoproj/argo-events/stable/manifests/install-validating-webhook.yaml 
kubectl apply -n argo-events -f https://raw.githubusercontent.com/argoproj/argo-events/stable/examples/eventbus/native.yaml

Create RBAC Policies:

kubectl apply -n argo-events -f https://raw.githubusercontent.com/argoproj/argo-events/master/examples/rbac/sensor-rbac.yaml 
kubectl apply -n argo-events -f https://raw.githubusercontent.com/argoproj/argo-events/master/examples/rbac/workflow-rbac.yaml

Verify Argo Events component:
```
kubectl get all -n argo-events
```

Wait until all components are ready.

Step 5: Configure Event Sources and Sensors

Navigate to the event sources directory:
```
cd infra/argo-events/event-sources
```

Apply the webhook event source:

kubectl apply -f webhook-eventsource.yaml

Move to the pipeline templates:
```
cd ../../../pipelines/templates
```

Deploy the CI template:

kubectl apply -f node-api-ci-template.yaml

Navigate to the sensors directory:
```
cd ../../infra/argo-events/sensors
```
Apply the polling sensor:
```
kubectl apply -f polling-sensor.yaml
```

Step 6: Build and Deploy Custom Poller

Navigate to the custom poller directory:
```
cd ../custom-poller
```

Build and push the poller image:

docker build --platform linux/amd64 -t <your-docker-registry>/github-poller:0.0.1 .

Deploy the poller:
```
kubectl apply -f poller-cronjob.yaml
```

Test the workflow

To test the changes make a simple change in the demo nodejs app. You can change the Version in the index.ts as it doesn't modify the code but will still trigger the workflow

Wrapping Up

You’ve now set up Continuous Integration (CI) using Argo Workflows and Argo Events. You can view workflows in the Argo Workflows UI by visiting:

https://localhost:2746/workflows/argo-events?&limit=50