Admission Controllers in Action: Datree's Approach

In the eighth part, the author talked about admission controllers. In this, the ninth, we will see how ACs can be used for practical purposes.

At the same time, it may be considered that this is the second part of the review, so both of the parts will be marked with the appropriate #datree tag.

The originality of Datree's approach

In brief, Datree's integration enables you to check your resources against the defined policy a moment before you put them into a cluster... by leveraging an admission webhook! 😎

The webhook implemented with ValidatingWebhookConfiguration will detect operations such as CREATE, UPDATE or DELETE, and it will start a policy check against the configs related to each operation.

If any configuration errors are discovered, the webhook will refuse the action and show a thorough output with guidance on how to fix each error.

Every cluster operation that is tied up once the webhook is installed will cause a Datree policy check. If there are no configuration errors, the resource will get a green light🚦 to be applied or updated.

🔬 Datree is functioning well in a full-scale cluster and also in a k3s/k3d-baked one! It make a debugging process more suitable even for local development.

Let's go step by step

Following the Software-as-a-Service paradigm, Datree provides their users access to the misconfigurations' database and to the personal workspace at their website, where all the checks initiated by the user become aggregated.

This cheeky astronaut design will brighten up your day.

🎯 Step 1. Access token

They [want to] know everything about you! Well, relax, it is a joke. Sign up or log in, then grab your token to access datree programmatically. API access tokens are widespread in 2022, aren't they? 🔏

💰 There is only one token available when using the so-called Free Plan, enough for evaluation purposes (up to 4 Kubernetes nodes are supported; service access logs will be stored for two weeks).

🎯 Step 2. Set up your CLI environment

The following binaries must be installed on the machine: kubectl, openssl (required for creating a certificate authority, CA) and curl.

Assume everything is in place. Let's run:

$ DATREE_TOKEN=[your-token] bash <(curl https://get.datree.io/admission-webhook)

What you should see and what should happen to your cluster (yes, API requests are additionally encrypted):

🔑 Generating TLS keys...
Generating a RSA private key
Signature ok
subject=CN = webhook-server.datree.svc
Getting CA Private Key
/home/roman
🔗 Creating webhook secret tls...
secret "webhook-server-tls" deleted
secret/webhook-server-tls created
🔗 Creating core resources...
serviceaccount/webhook-server-datree created
clusterrolebinding.rbac.authorization.k8s.io/rolebinding:webhook-server-datree created
clusterrole.rbac.authorization.k8s.io/webhook-server-datree created
deployment.apps/webhook-server configured
service/webhook-server created
deployment "webhook-server" successfully rolled out
🔗 Creating validation webhook resource...
validatingwebhookconfiguration.admissionregistration.k8s.io/webhook-datree configured
🎉 DONE! The webhook server is now deployed and configured

🎯 Step 3. Protect your access token

Because your token is private and you don't want to store it in your repository, we advise setting or changing it using a different kubectl patch command:

$ kubectl patch deployment webhook-server -n datree -p '
spec:
  template:
    spec:
      containers:
        - name: server
          env:
            - name: DATREE_TOKEN
              value: "<your-token>"'

🎯 Step 4. Deploy something: magic will work for you

The author does not want to reinvent the wheel:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

Let's focus and look at the kubectl apply -f nginx-deployment.yaml routine's result (the deployment has been denied by AC):

$ kubectl apply -f nginx-deployment.yaml
Error from server: error when creating "nginx-deployment.yaml": admission webhook "webhook-server.datree.svc" denied the request: 
webhook-nginx-deployment-Deployment.tmp.yaml

[V] YAML validation
[V] Kubernetes schema validation

[X] Policy check

❌  Ensure each container has a configured CPU limit  [1 occurrence]
    - metadata.name: nginx-deployment (kind: Deployment)
💡  Missing property object `limits.cpu` - value should be within the accepted boundaries recommended by the organization

❌  Ensure each container has a configured CPU request  [1 occurrence]
    - metadata.name: nginx-deployment (kind: Deployment)
💡  Missing property object `requests.cpu` - value should be within the accepted boundaries recommended by the organization

❌  Ensure each container has a configured liveness probe  [1 occurrence]
    - metadata.name: nginx-deployment (kind: Deployment)
💡  Missing property object `livenessProbe` - add a properly configured livenessProbe to catch possible deadlocks

❌  Ensure each container has a configured memory limit  [1 occurrence]
    - metadata.name: nginx-deployment (kind: Deployment)
💡  Missing property object `limits.memory` - value should be within the accepted boundaries recommended by the organization

❌  Ensure each container has a configured memory request  [1 occurrence]
    - metadata.name: nginx-deployment (kind: Deployment)
💡  Missing property object `requests.memory` - value should be within the accepted boundaries recommended by the organization

❌  Ensure each container has a configured readiness probe  [1 occurrence]
    - metadata.name: nginx-deployment (kind: Deployment)
💡  Missing property object `readinessProbe` - add a properly configured readinessProbe to notify kubelet your Pods are ready for traffic

❌  Prevent workload from using the default namespace  [1 occurrence]
    - metadata.name: nginx-deployment (kind: Deployment)
💡  Incorrect value for key `namespace` - use an explicit namespace instead of the default one (`default`)


(Summary)

- Passing YAML validation: 1/1

- Passing Kubernetes (v1.21.5) schema validation: 1/1

- Passing policy check: 0/1

+-----------------------------------+-----------------------+
| Enabled rules in policy "Default" | 21                    |
| Configs tested against policy     | 1                     |
| Total rules evaluated             | 21                    |
| Total rules skipped               | 0                     |
| Total rules failed                | 7                     |
| Total rules passed                | 14                    |
| See all rules in policy           | https://app.datree.io |
+-----------------------------------+-----------------------+

Opening the link, you'll be redirected to your personal workspace.

🎯 Step 5. Is such rigor necessary?

You can audit reactive policies and review invocation history. If checks are too strict, unset some of the policies.

For example, not in every deployment you really need pre-configured container readiness probes [or CPU & memory limits].

Well, another tryout will be on edited YAML (Octopus may be your fellow):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  selector:
    matchLabels:
       app: nginx
  replicas: 2
  strategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:1.14.2
          ports:
            - containerPort: 80
          resources:
            requests:
              memory: 100Mi
              cpu: 100m
            limits:
              memory: 200Mi
              cpu: '1'

As we really want to use the default namespace and have no fears with it, let's disable Prevent workload from using the default namespace policy in Datree web UI.

Also we may want to liberate us from Ensure each container has a configured readiness probe and Ensure each container has a configured liveness probe policies.

Et voilà! 🎭

$ kubectl apply -f nginx-deployment-advanced.yaml
deployment.apps/nginx-deployment created

Now all checks are passed successfully and the admission controller have got us a green light🚦 and allowed the deployment!

🎯 Step 6. Hey, do not climb where it is not necessary

If you want datree to disregard a namespace, add the label admission.datree/validate=skip to its configuration.

$ kubectl label namespaces default "admission.datree/validate=skip"

How to wipe traces

To delete the label and resume running the datree webhook on the namespace again:

$ kubectl label namespaces default "admission.datree/validate-"

Uninstall the webhook

Copy the following command and run it in your terminal to remove the webhook:

$ bash <(curl https://get.datree.io/admission-webhook-uninstall)
validatingwebhookconfiguration.admissionregistration.k8s.io "webhook-datree" deleted
service "webhook-server" deleted
deployment.apps "webhook-server" deleted
secret "webhook-server-tls" deleted
clusterrolebinding.rbac.authorization.k8s.io "rolebinding:webhook-server-datree" deleted
serviceaccount "webhook-server-datree" deleted
clusterrole.rbac.authorization.k8s.io "webhook-server-datree" deleted
namespace/kube-system unlabeled
namespace "datree" deleted

Summing up what was said

As you could understand, the possibilities of Kubernetes API are quite wide. The author hopes that he not only prepared an overview of a useful solution, but also explained the theoretical aspects of its functionality.

Blog