Kubectl Patch: What You Can Use It for and How to Do It

By Eric Goebelbecker

Kubernetes' automated deployments make life easier. Managing integrated applications used to require multiple systems, with error-prone orchestration that crossed multiple computer and application boundaries. But with k8s, you can define your application as deployments and let the orchestrator do the rest.

But sometimes you need to change your configuration, and you don't want to start out with a new deployment. Maybe you want to test a new configuration in development. Or, perhaps you need to deploy a change to production fast because of an emergency. Fortunately, Kubernetes has tools for this, too.

Kubectl patch is a tool for updating resources during runtime. It has several important nuances and isn't always the best way to perform a configuration update. Let's look at how it works, when to use it, and how it compares to apply, edit, and replace.

What is the Kubectl Patch Command?

Kubectl

First, let's take a quick look at kubectl. It's a command-line tool for managing Kubernetes clusters. You can use it to check cluster status, apply deployments, alter resources, and just about anything else you need to do to keep your containers running.

You may need to install kubetctl separately., or it the packages that install Docker and Kubernetes for your operating system may do it for you.

Kubectl Patch

Patch is a command line option for updating Kubernetes API objects. You can use it to update a running configuration. You do this by supplying it with the section to update, instead of a completely new configuration, as you would with kubectl apply.

Kubectl patch supports three patch types:

strategic - a strategic patch will merge the configuration you supply with the existing one based on the node's type.
JSON merge - this type of patch follows the algorithm specified in RFC 7386.
JSON - with this patch, you specify the operation you want kubectl to perform on each configuration node.

Each type has its own semantics, advantages, and disadvantages. We'll look at examples of each below.

Use of Kubectl Patch Considered Harmful?

Before we get started with the examples, let's discuss why patch isn't the best option for modifying Kubernetes configurations, especially in production.

When you patch an API object, you're changing its runtime configuration. But, unless you update the configuration you used to create it, it falls out of sync. As a result, the runtime configuration no longer matches the files you used to create the object.

So, you either need to track patches or devise a way to update your "master" configuration.

As a result, kubectl apply is often a better option. When you use it, you supply kubectl with all the required fields to create the object. So, your copy of the configuration stays in sync with your archived copies. Hopefully, you're using Infrastructure as Code to manage your cluster configurations.

Kubectl Patch Examples

Preparation

If you want to follow the examples in this article, you'll need a Kubernetes cluster and kubectl configured so that it has proper access. If you want to work on your desktop, a system running Docker with Minikube installed and started will work fine, too.

First, deploy a simple Nginx deployment with this YAML configuration:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80

Deploy it with the apply command:

kubectl apply -f nginx_deployment.yaml
deployment.apps/nginx-deployment created
% kubectl get pods --show-labels
NAME                                READY   STATUS    RESTARTS   AGE   LABELS
nginx-deployment-544dc8b7c4-46tkh   1/1     Running   0          5s    app=nginx,pod-template-hash=544dc8b7c4
nginx-deployment-544dc8b7c4-sjfz5   1/1     Running   0          5s    app=nginx,pod-template-hash=544dc8b7c4

We're ready to go.

Strategic Patch

Let's start by applying a patch with the default mode: strategic.

Start by retrieving detailed information about the nginx containers we're running. This command retrieves the current pods and parses out the container types, then it filters for names that include nginx.

kubectl get pods -o jsonpath='{range .items[*]}{"
"}{.metadata.name}{":   "}{range .spec.containers[*]}{.image}{", "}{end}{end}' |grep nginx
nginx-deployment-544dc8b7c4-7hjdd:    nginx:latest,
nginx-deployment-544dc8b7c4-sdffc:    nginx:latest,

As expected, we're running two instances of nginx's latest image.

Let's patch the deployment with a specific version of the container.

Start by creating a new YAML file named nginx_patch.yaml:

spec:
  template:
    spec:
      containers:
      - name: nginx
        image: nginx:1.23.0-alpine

This file contains just enough information for patch to figure out what we're trying to do: replace image: nginx:latest with image: nginx:1.23.0-alpine.

Let's apply this patch with kubectl patch:

kubectl patch deployment nginx-deployment --patch-file nginx_patch.yaml
deployment.apps/nginx-deployment patched

We have to tell kubectl which deployment we're patching, along with the file to apply.

Wait a few seconds, and then retrieve the container details again.

kubectl get pods -o jsonpath='{range .items[*]}{"
"}{.metadata.name}{":   "}{range .spec.containers[*]}{.image}{", "}{end}{end}' |grep nginx
nginx-deployment-5d985b54cc-j8lpv:    nginx:1.23.0-alpine,
nginx-deployment-5d985b54cc-lf5wq:    nginx:1.23.0-alpine,

Kubectl updated the configuration and restarted the containers for us!

We can retrieve the YAML configuration for one of the pods and see the change:

kubectl get pod nginx-deployment-5d985b54cc-j8lpv -o yaml
apiVersion: v1
kind: Pod
metadata:
spec:
  containers:
  - image: nginx:1.23.0-alpine
    imagePullPolicy: Always
    name: nginx
    ports:
    - containerPort: 80
      protocol: TCP

With a strategic patch, kubectl does the work of figuring out how to apply the changes we supplied it. Here, we replaced a node in a list with a new value. Depending on the change and the underlying type, the changes can get more complicated, so the default merge type is often your best option.

Merge

With a merge patch, kubectl will replace the entire targeted configuration node with the supplied code. This patch type is often called a JSON merge, even though you can supply it with YAML configuration, and it doesn't merge the target configuration node: it replaces it. This is because it follows the JSON merge semantics defined in RFC 7386.

Let's look at an example.

We want to add Redis to our deployment. So, here's a file with a Redis container in the containers list. It's in a file name nginx_merge_path.yaml:

spec:
  template:
    spec:
      containers:
      - name: redis
        image: redis:latest
        ports:
        - containerPort: 6379

Apply this patch as a merge with the --type command line argument.

kubectl patch deployment nginx-deployment --type merge --patch-file nginx_merge_patch.yaml
deployment.apps/nginx-deployment patched
% kubectl get pods -owide
NAME                                READY   STATUS    RESTARTS   AGE   IP           NODE       NOMINATED NODE   READINESS GATES
nginx-deployment-7c99566855-4t5g7   1/1     Running   0          12s   172.17.0.5   minikube   <none>           <none>
nginx-deployment-7c99566855-f2s2c   1/1     Running   0          15s   172.17.0.6   minikube   <none>           <none>
% kubectl get pods -o jsonpath='{range .items[*]}{"
"}{.metadata.name}{":   "}{range .spec.containers[*]}{.image}{", "}{end}{end}' |grep nginx
nginx-deployment-7c99566855-4t5g7:    redis:latest,
nginx-deployment-7c99566855-f2s2c:    redis:latest,

The nginx containers are gone? Kubectl removed them and replaced them with Redis!

Let's look at the runtime config for one pod:

kubectl get pod nginx-deployment-7c99566855-4t5g7 -o yaml |less
spec:
  containers:
  - image: redis:latest
    imagePullPolicy: Always
    name: redis
    ports:
    - containerPort: 6379
      protocol: TCP

Yes, that's exactly what the configuration shows. JSON merge replaced the containers list with the new one.

What happens when we reapply the previous patch with the default type?

kubectl patch deployment nginx-deployment --patch-file nginx_patch.yaml
deployment.apps/nginx-deployment patched

kubectl get pods -owide
NAME                                READY   STATUS        RESTARTS   AGE     IP           NODE       NOMINATED NODE   READINESS GATES
nginx-deployment-86c4594d8-ht4zv    2/2     Running       0          2s      172.17.0.6   minikube   <none>           <none>
nginx-deployment-86c4594d8-jmlnx    2/2     Running       0          4s      172.17.0.7   minikube   <none>           <none>

kubectl get pods --all-namespaces -o jsonpath='{range .items[*]}{"
"}{.metadata.name}{":   "}{range .spec.containers[*]}{.image}{", "}{end}{end}' |grep nginx
nginx-deployment-86c4594d8-ht4zv: nginx:1.23.0-alpine, redis:latest,
nginx-deployment-86c4594d8-jmlnx: nginx:1.23.0-alpine, redis:latest,
% kubectl get pod nginx-deployment-86c4594d8-jmlnx -o yaml |less
spec:
  containers:
  - image: nginx:1.23.0-alpine
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: kube-api-access-4flnp
      readOnly: true
  - image: redis:latest
    imagePullPolicy: Always
    name: redis
    ports:
    - containerPort: 6379
      protocol: TCP

The Nginx containers are back.

So, we've seen the difference between strategic and merge patch types at work. The merge replaced the containers list, strategic added the changes to the list instead.

JSON Patch

A JSON patch follows the methodology defined in RFC 6902. Instead of supplying kubectl with configuration information, you give it a list of directives.

Let's roll the Nginx containers back to the latest image:

- op: replace
  path: "/spec/template/spec/containers/0/image"
  value: nginx:latest

This code tells kubectl to find the node at /spec/template/spec/containers/0/image and replace its value with the new image tag.

That JSON path will take us to the image node in the first item in the containers list.

Let's run the patch:

kubectl patch deployment nginx-deployment --type JSON --patch-file nginx_json_patch.yaml
deployment.apps/nginx-deployment patched

kubectl get pods -o jsonpath='{range .items[*]}{"
"}{.metadata.name}{":   "}{range .spec.containers[*]}{.image}{", "}{end}{end}' |grep nginx
nginx-deployment-9b7747f45-4svqm:    nginx:latest, redis:latest,
nginx-deployment-9b7747f45-hqjgh:    nginx:latest, redis:latest,

That did it! We have two new pods and they're running the latest Nginx and Redis containers now.

Patch vs. Edit vs. Apply vs. Replace

Patch has three flexible methods for changing API objects, but it's not the only option.

Kubectl edit is an interactive way to change configuration.

This command:

kubectl edit deployment nginx-deployment

Brings up an editor with the nginx-deployment configuration. The default editor is vi or vim, but you can override it with the EDITOR environment variable.

Here's a trimmed copy of the edit window in vim:

# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "5"
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{},"labels":{"app":"nginx"},"name":"nginx-deployment","namespace":"default"},"spec":{"replicas":2,"selector":{"matchLabels":{"app":"nginx"}},"template":{"metadata":{"labels":{"app":"nginx"}},"spec":{"containers":[{"image":"nginx:latest","name":"nginx","ports":[{"containerPort":80}]}]}}}}
  creationTimestamp: "2022-07-14T21:26:46Z"
  generation: 7
  labels:
    app: nginx
  name: nginx-deployment
  namespace: default
  resourceVersion: "355162"
  uid: 38dd5518-d4b3-433c-9371-31780ede214a

If you change the file, kubectl applies the new version when you save it.

So, the biggest difference between edit and patch is that edit is interactive. With it, you view the current configuration of a single object, change it in line, and then save it. Patching is a batch operation. It's well-suited for running from shell scripts. As a result, you can use it to change multiple objects at once via those scripts.

Apply is also a batch tool. But even if you are using it to change an existing object, you need to supply it with all the required fields for the object. But, you can use kubectl get -o yaml, edit the results, and then apply the new configuration. The advantage of this approach is that you can save the new configuration and archive it for disaster recovery or even commit it to source control.

Replace goes one step further than apply: it requires the entire configuration, including the fields with defaults that apply does not require. Similar to apply though, replace is a good option since you finish the change with a complete copy of the API object.

Kubectl Patch

In this article, we covered what kubectl patch is and how to use it. We covered examples of the three patch types and saw how they differ in how they apply changes to runtime configuration. Then, we wrapped up with a quick comparison between patch and other methods for modifying Kubernetes API objects.

Not that you're familiar with patch, you can add it to your workflow when and if it makes sense.

This post was written by Eric Goebelbecker. Eric has worked in the financial markets in New York City for 25 years, developing infrastructure for market data and financial information exchange (FIX) protocol networks. He loves to talk about what makes teams effective (or not so effective!).