We are in an era where digital solutions are more and more present in our everyday life, but all this digital world (especially all the tooling behind) is a source of pollution.

As you can sort you waste or reuse a lot of things, we will see how you can reduce your carbon print (and your cloud provider billing) with your Kubernetes cluster!

Kube-Green

Kube-Green is an operator that can scale down your deployments to 0 and avoid your CronJobs to be triggered outside your "working days".

The objective is simple, shutdown your pods and jobs that can run during the night or during weekends, before restarting them in the morning of your working days. So a lot of process are not running "for nothing", generating CO2 and it can scale down the number of nodes of your cluster if you are on AWS. (The money saving part)

When can I use it?

For sure, if you have a 24/7 service in your production, you should not use it in this environment. If your production is only used during particular time frames, it can be used in this case.

In general this operator is more oriented for dev and/or tests environments, which are generally always up for no reasons.

Is it possible to exclude some processes?

It's a good question because you may need to keep particular jobs or pods even in a dev environment. So it's possible and we will see later how we can do it.

Serup

All the following informations about the setup is coming from the KubeGreen documentation. So don't hesitate to check it there directly to have the latest version of the setup documentation.

Prerequirements

On this day (early june 2023) KubeGreen works with Kubernetes 1.19 to 1.26 and OpenShift v4.

Also you need to have cert-manager to be installed on your cluster.

Installation

Several ways are offered by KubeGreen, but we will only use kubectl apply in this post.

If you want the latest version of KubeGreen, use the following command

kubectl apply -f https://github.com/kube-green/kube-green/releases/latest/download/kube-green.yaml

But if you want a precise version of KubeGreen, use the following command and replace ${RELEASE_TAG} by the version number you want.

kubectl apply -f https://github.com/kube-green/kube-green/releases/download/${RELEASE_TAG}/kube-green.yaml

*/!\ Warning : If you want to use it in the cloud /!\*
The documentation says that you can encounter several issues if you are using KubeGreen in GKE or AWS. So if you are in this case, go check the documentation before going forward.

Configuration

Once the operator deployed, you can create a SleepInfo instance that will define which resources must be stopped, when they should be stopped and when they must be restarted.

You must know that SleepInfo only act in the namespace where it's deployed. It means that you must deploy a instance of SleepInfo in each namespace you want to use KubeGreen. (It will avoid to shutdown necessary deployments for the proper functionning of the cluster)

apiVersion: kube-green.com/v1alpha1  
kind: SleepInfo  
metadata:  
    name: working-hours  
spec:  
    weekdays: "1-5"  
    sleepAt: "20:00"  
    wakeUpAt: "08:00"  
    timeZone: "Europe/Rome"

In this example, which is the simplest use case of KubeGreen, all the deployment in the namespace will be shutdown at 20:00 (Rome timezone) from monday to friday, and they will be restarted from monday to friday at 08:00 (Rome timezone).

To have a better understanding about SleepInfo, we will check every field and then see differents use cases.

SleepInfo : Fields

• weekdays : Define week days where the operator must act.
  • 1 is for monday
  • 1-5 is from monday to friday
  • 2-6 is from tuesday to saturday
  • * is for everyday
• sleepAt : Define with the HH:mm pattern, it will define the time when resources must be stopped. (ex: 19:00, 08:53: 23:45)
• wakeUpAt : [Optionnal] Define with the HH:mm pattern, it will define the time when resources must be restarted. (ex: 19:00, 08:53: 23:45)
• timeZone : [Optionnal] Define the timezone related to times defined previously
  • This value must match a timezone name listed in the IANA specification. (TZ identifier column in Wikipedia - Timezones)
  • Default value is UTC.
• suspendDeployments : [Optionnal] Boolean to let the operator knows if it must shutdown deployments in this namespace.
  • Default value is true.
• suspendCronJobs : [Optionnal] Boolean to let the operator knows if it must shutdown cronjobs in this namespace.
  • Default value is false
• excludeRef : [Optionnal] Object array containing references to deployments and/or cronjobs to exclude.
  • apiVersion : Resource version to exclude. Actually supported : apps/v1, batch/v1beta1 and batch/v1 (don't use this property if you are using matchLabels)
  • kind : Kind of resource to exclude. Actually supported : Deployment and CronJob (don't use this property if you are using matchLabels)
  • name : Name of the resource to exclude (don't use this property if you are using matchLabels)
  • matchLabels : Label list to identify resources to exclude (don't use this property if you are using name, apiVersion and kind)

Use cases

Stop everything except api-gateway deployment

apiVersion: kube-green.com/v1alpha1
kind: SleepInfo
metadata:
  name: working-hours
spec:
  weekdays: "1-5"
  sleepAt: "20:00"
  wakeUpAt: "08:00"
  timeZone: "Europe/Rome"
  suspendCronJobs: true
  excludeRef:
    - apiVersion: "apps/v1"
      kind:       Deployment
      name:       api-gateway  

In this example, we can see that we want to shutdown everything (deployments & cronjobs) at 20:00 from monday to friday and restart them at 08:00 from monday to friday, except "api-gateway" deployment.

Useful if you only have few resources to exclude.

Shutdown everything except the one with a particular label

apiVersion: kube-green.com/v1alpha1
kind: SleepInfo
metadata:
  name: working-hours
spec:
  weekdays: "1-5"
  sleepAt: "20:00"
  wakeUpAt: "08:00"
  timeZone: "Europe/Rome"
  suspendCronJobs: true
  excludeRef:
    - matchLabels: 
        kube-green.dev/exclude: true

In this example, we can see that everything will be shutdown except resources with the label kube-green.dev/exclude: true.

You must use it if you have a lot of resources to exclude.

Only stop CronJobs

apiVersion: kube-green.com/v1alpha1
kind: SleepInfo
metadata:
  name: working-hours
spec:
  weekdays: "*"
  sleepAt: "20:00"
  wakeUpAt: "08:00"
  suspendCronJobs: true
  suspendDeployments: false

To stop only all the CronJobs, you must set suspendCronJobs to true and suspendDeployments to false.

Without restart

apiVersion: kube-green.com/v1alpha1
kind: SleepInfo
metadata:
  name: working-hours
spec:
  weekdays: "1-5"
  sleepAt: "20:00"
  timeZone: "Europe/Rome"
  suspendCronJobs: true
  excludeRef:
    - apiVersion: "apps/v1"
      kind:       Deployment
      name:       api-gateway

This example is useful to let us know that defining the wakeUpAt value is mendatory. You can ask in which case it can be useful, and there are a simple case : test environments.

Depending on your context, you can have a testing and/or load testing environment which is used once in a month or less. In this case, restarting the environment during weeks without tests is useless. And you will still be able to update the SleepInfo instance to restart all the pods when needed.

Links

• Kube-Green Website
• Kube-Green - Github project

Conclusion

KubeGreen is a promising project which, I hope, will help us to have less impact on our planet. I think about stopping useless daemonsets, interact with other operators (like Crossplane to shutdown resources in the cloud)...

May be it seems nothing, but like everything else, if we are all doing little efforts here and there, it will make something big!

I hope it will help you and if you have any questions (there are not dumb questions) or some points are not clear for you, don't hesitate to add your question in the comments or to contact me directly on LinkedIn.