TheLazy.Dev
Portfolio GitHub

Blog

HULL Tutorial 06: Writing Services and Ingresses

gre9ory

gre9ory

Posted on May 31, 2022

HULL Tutorial 06: Writing Services and Ingresses

Preparation

Next up is a closer look at the objects from the Kubernetes Service API which includes Services, Ingresses and Endpoints.

But as before copy over our working chart from the last tutorial part to a new folder:

cd ~/kubernetes-dashboard-hull && cp -R 05_workloads/ 06_service_api && cd 06_service_api/kubernetes-dashboard-hull
Enter fullscreen mode Exit fullscreen mode

To remove the HULL hull.objects from values.full.yaml and prepare our testing values.yaml enter:

sed '/  objects:/Q' values.full.yaml > values.yaml
Enter fullscreen mode Exit fullscreen mode

and verify the result:

cat values.yaml
Enter fullscreen mode Exit fullscreen mode

looks like:

metrics-server:
  enabled: false
hull:
  config:
    specific:
      protocolHttp: false
      rbac:
        clusterReadOnlyRole: false
        clusterRoleMetrics: true
      settings: {}
      pinnedCRDs: {}
Enter fullscreen mode Exit fullscreen mode

Check for existing Service API objects

Start the investigation as you did before:

find ../kubernetes-dashboard/templates -type f -iregex '.*\(Service\|\Ingress\|\Endpoint\).*' | sort
Enter fullscreen mode Exit fullscreen mode

which returns:

../kubernetes-dashboard/templates/ingress.yaml
../kubernetes-dashboard/templates/service.yaml
../kubernetes-dashboard/templates/serviceaccount.yaml
../kubernetes-dashboard/templates/servicemonitor.yaml
Enter fullscreen mode Exit fullscreen mode

Ignoring the ServiceAccount (already handled) and the ServiceMonitor (handled later) there exists a Service and an Ingress definition.

Defining Services in HULL

The Service template contents of /templates/service.yaml:

cat ../kubernetes-dashboard/templates/service.yaml
Enter fullscreen mode Exit fullscreen mode

do look like this:

# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: v1
kind: Service
metadata:
  name: {{ template "kubernetes-dashboard.fullname" . }}
  labels:
    {{ include "kubernetes-dashboard.labels" . | nindent 4 }}
    app.kubernetes.io/component: kubernetes-dashboard
    {{ .Values.service.clusterServiceLabel.key | nindent 4}}: {{ .Values.service.clusterServiceLabel.enabled | quote }}
    {{- if .Values.service.labels }}
    {{ toYaml .Values.service.labels | nindent 4 }}
    {{- end }}
    {{- if .Values.commonLabels }}
    {{- include "common.tplvalues.render" ( dict "value" .Values.commonLabels "context" $ ) | nindent 4 }}
    {{- end }}
  annotations:
    {{- if .Values.commonAnnotations }}
    {{- include "common.tplvalues.render" ( dict "value" .Values.commonAnnotations "context" $ ) | nindent 4 }}
    {{- end }}
    {{- if .Values.service.annotations }}
    {{ toYaml .Values.service.annotations | nindent 4 }}
    {{- end }}
spec:
  type: {{ .Values.service.type }}
  ports:
  - port: {{ .Values.service.externalPort }}
{{- if .Values.protocolHttp }}
    targetPort: http
    name: http
{{- else }}
    targetPort: https
    name: https
{{- end }}
{{- if hasKey .Values.service "nodePort" }}
    nodePort: {{ .Values.service.nodePort }}
{{- end }}
{{- if .Values.service.loadBalancerSourceRanges }}
  loadBalancerSourceRanges:
{{ toYaml .Values.service.loadBalancerSourceRanges | nindent 4 }}
{{- end }}
  selector:
{{ include "kubernetes-dashboard.matchLabels" . | nindent 4 }}
    app.kubernetes.io/component: kubernetes-dashboard
{{- if .Values.service.loadBalancerIP }}
  loadBalancerIP: {{ .Values.service.loadBalancerIP }}
{{- end }}
Enter fullscreen mode Exit fullscreen mode

Again there is a reference to the .Values.protocolHttp property which now exists in the hull.config.specific section. The ports' name and targetPort are set dependent on protocolHttp while the externalPort is centrally defined. Therefore it makes sense to centralize access to externalPort under hull.config.specific, even more so because the externalPort is also accessed when defining the Ingress (more on that later). To do so execute:

sed '/^\s\s\s\sspecific/r'<(
      echo "      externalPort: 443"
    ) -i -- values.yaml
Enter fullscreen mode Exit fullscreen mode

Apart from the ports there is only the selector worth discussing, the rest of the Service template is straightforward 1:1 mappings. But also check on the service block defaults in the kubernetes-dashboard's values.yaml with:

cat ../kubernetes-dashboard/values.yaml | grep "^service:" -B 1 -A 23
Enter fullscreen mode Exit fullscreen mode

which prints:


service:
  type: ClusterIP
  # Dashboard service port
  externalPort: 443

  ## LoadBalancerSourcesRange is a list of allowed CIDR values, which are combined with ServicePort to
  ## set allowed inbound rules on the security group assigned to the master load balancer
  # loadBalancerSourceRanges: []

  ## A user-specified IP address for load balancer to use as External IP (if supported)
  # loadBalancerIP:

  ## Additional Kubernetes Dashboard Service annotations
  annotations: {}

  ## Here labels can be added to the Kubernetes Dashboard service
  labels: {}

  ## Enable or disable the kubernetes.io/cluster-service label. Should be disabled for GKE clusters >=1.15.
  ## Otherwise, the addon manager will presume ownership of the service and try to delete it.
  clusterServiceLabel:
    enabled: true
    key: "kubernetes.io/cluster-service"
Enter fullscreen mode Exit fullscreen mode

The label kubernetes.io/cluster-service is added to the Service object and the enabled value serves as the value, this is represented in this part of the Service template:

    {{ .Values.service.clusterServiceLabel.key | nindent 4}}: {{ .Values.service.clusterServiceLabel.enabled | quote }}
Enter fullscreen mode Exit fullscreen mode

While you could mimic the logic and move the clusterServiceLabel to the hull.config.specific section it does not offer any advantage over adding the label to the new service instance with the default value of true. A comment may be added of course to explain the deeper meaning of this label.

A closer look at selectors of Services

As previously highlighted, the selector matchLabels for workloads are automatically constructed from the following labels:

  • app.kubernetes.io/component: default
  • app.kubernetes.io/instance: release-name
  • app.kubernetes.io/name: kubernetes-dashboard

The Service object's selector's in HULL follow the same construction principle, so this means that any Service whose key is identical to a workload object instance the selector will match and the Service will front the pods of the workload. For our example, if you define either a Deployment, DaemonSet or StatefulSet with key dashboard, any Service object with the same key dashboard will match and pose as the Service object to the pods on the network level. This is what you want to achieve here, a Service for the dashboard Deployment. On a side note, for Jobs the selector property is omitted automatically by HULL because in this case the selector handling of Jobs is internally managed by Kubernetes.

So if you leave out the selector specification you will automatically have the default selector created but if you want to model the selector property yourself you can overwrite it explicitly. A use case for this is e.g. the creation of an additional headless Service for StatefulSets or any other more finegrained controlling of Service to Pod matching. For this there also exists a HULL transformation, the hull.util.transformation.selector or short _HT&, which will be put to use later on.

But for now here is the converted Service object for the dashboard employing the techniques that you already have learned:

echo '  objects:
    service:
      dashboard:
        labels:
          ## Enable or disable the kubernetes.io/cluster-service label. Should be disabled for GKE clusters >=1.15.
          ## Otherwise, the addon manager will presume ownership of the service and try to delete it.
          "kubernetes.io/cluster-service": "true"
        type: ClusterIP
        ports:
          http:
            enabled: _HT?(index . "$").Values.hull.config.specific.protocolHttp
            port: _HT*hull.config.specific.externalPort
            targetPort: http
          https:
            enabled: _HT?(not (index . "$").Values.hull.config.specific.protocolHttp)
            port: _HT*hull.config.specific.externalPort
            targetPort: https' >> values.yaml
Enter fullscreen mode Exit fullscreen mode

Defining Ingresses for inbound traffic

Next take a look at the original ingress definition with:

cat ../kubernetes-dashboard/templates/ingress.yaml
Enter fullscreen mode Exit fullscreen mode

returning:

# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

{{ if .Values.ingress.enabled -}}
{{- $serviceName := include "kubernetes-dashboard.fullname" . -}}
{{- $servicePort := .Values.service.externalPort -}}
{{- $paths := .Values.ingress.paths -}}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: {{ template "kubernetes-dashboard.fullname" . }}
  labels:
    {{- include "kubernetes-dashboard.labels" . | nindent 4 }}
    {{- range $key, $value := .Values.ingress.labels }}
    {{ $key }}: {{ $value | quote }}
    {{- end }}
    {{- if .Values.commonLabels }}
    {{- include "common.tplvalues.render" ( dict "value" .Values.commonLabels "context" $ ) | nindent 4 }}
    {{- end }}
  annotations:
    {{- if .Values.commonAnnotations }}
    {{- include "common.tplvalues.render" ( dict "value" .Values.commonAnnotations "context" $ ) | nindent 4 }}
    {{- end }}
    {{- if not .Values.protocolHttp }}
    # Add https backend protocol support for ingress-nginx
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    # Add https backend protocol support for GKE
    service.alpha.kubernetes.io/app-protocols: '{"https":"HTTPS"}'
    {{- end }}
    {{- with .Values.ingress.annotations }}
    {{ toYaml . | nindent 4 }}
    {{- end }}
spec:
  {{- with .Values.ingress.className }}
  ingressClassName: {{ . | quote }}
  {{- end }}
  rules:
  {{- if .Values.ingress.hosts }}
  {{- range $host := .Values.ingress.hosts }}
    - host: {{ $host }}
      http:
        paths:
  {{- if len ($.Values.ingress.customPaths) }}
  {{- "\n" }}{{ tpl (toYaml $.Values.ingress.customPaths | nindent 10) $ }}
  {{- else }}
  {{- range $p := $paths }}
          - path: {{ $p }}
            pathType: ImplementationSpecific
            backend:
              service:
                name: {{ $serviceName }}
                port:
                  number: {{ $servicePort }}
  {{- end -}}
  {{- end -}}
  {{- end -}}
  {{- else }}
    - http:
        paths:
  {{- if len ($.Values.ingress.customPaths) }}
  {{- "\n" }}{{ tpl (toYaml $.Values.ingress.customPaths | nindent 10) $ }}
  {{- else }}
  {{- range $p := $paths }}
          - path: {{ $p }}
            pathType: ImplementationSpecific
            backend:
              service:
                name: {{ $serviceName }}
                port:
                  number: {{ $servicePort }}
  {{- end -}}
  {{- end -}}
  {{- end -}}
  {{- if .Values.ingress.tls }}
  tls:
{{ toYaml .Values.ingress.tls | nindent 4 }}
  {{- end -}}
{{- end -}}
Enter fullscreen mode Exit fullscreen mode

This is what the values.yaml has to say about the ingress definition:

cat ../kubernetes-dashboard/values.yaml | grep "^ingress:" -B 1 -A 59
Enter fullscreen mode Exit fullscreen mode

which is:


ingress:
  ## If true, Kubernetes Dashboard Ingress will be created.
  ##
  enabled: false

  ## Kubernetes Dashboard Ingress labels
  # labels:
  #   key: value

  ## Kubernetes Dashboard Ingress annotations
  # annotations:
  #   kubernetes.io/ingress.class: nginx
  #   kubernetes.io/tls-acme: 'true'

  ## If you plan to use TLS backend with enableInsecureLogin set to false
  ## (default), you need to uncomment the below.
  ## If you use ingress-nginx < 0.21.0
  #   nginx.ingress.kubernetes.io/secure-backends: "true"
  ## if you use ingress-nginx >= 0.21.0
  #   nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"

  ## Kubernetes Dashboard Ingress Class
  # className: "example-lb"

  ## Kubernetes Dashboard Ingress paths
  ## Both `/` and `/*` are required to work on gce ingress.
  paths:
    - /
  #  - /*

  ## Custom Kubernetes Dashboard Ingress paths. Will override default paths.
  ##
  customPaths: []
  #  - pathType: ImplementationSpecific
  #    backend:
  #      service:
  #        name: ssl-redirect
  #        port:
  #          name: use-annotation
  #  - pathType: ImplementationSpecific
  #    backend:
  #      service:
  #        name: >-
  #          {{ include "kubernetes-dashboard.fullname" . }}
  #        port:
  #          # Don't use string here, use only integer value!
  #          number: 443
  ## Kubernetes Dashboard Ingress hostnames
  ## Must be provided if Ingress is enabled
  ##
  # hosts:
  #   - kubernetes-dashboard.domain.com
  ## Kubernetes Dashboard Ingress TLS configuration
  ## Secrets must be manually created in the namespace
  ##
  # tls:
  #   - secretName: kubernetes-dashboard-tls
  #     hosts:
  #       - kubernetes-dashboard.domain.com
Enter fullscreen mode Exit fullscreen mode

The values.yaml documentation is a little confusing and the template quite difficult to decipher at once so break it apart to see what is happening here:

  • annotations are to be added to the ingress on the condition that protocolHttp is not true. Excerpt from ingress template: 
  {{- if not .Values.protocolHttp }}
    # Add https backend protocol support for ingress-nginx
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    # Add https backend protocol support for GKE
    service.alpha.kubernetes.io/app-protocols: '{"https":"HTTPS"}'
    {{- end }}
Enter fullscreen mode Exit fullscreen mode
  • the rules entries section is basically repeated in the template, one time for all the hosts if any are given or once without host property if no hosts are given. Regarding the paths to process in both blocks the template will write out all fully specified customPaths if any are provided or will iterate over the predefined paths if no customPaths are given where the paths all point to the dashboard Service: 
  rules:
  {{- if .Values.ingress.hosts }}
  {{- range $host := .Values.ingress.hosts }}
    - host: {{ $host }}
      http:
        paths:
  {{- if len ($.Values.ingress.customPaths) }}
  {{- "\n" }}{{ tpl (toYaml $.Values.ingress.customPaths | nindent 10) $ }}
  {{- else }}
  {{- range $p := $paths }}
          - path: {{ $p }}
            pathType: ImplementationSpecific
            backend:
              service:
                name: {{ $serviceName }}
                port:
                  number: {{ $servicePort }}
  {{- end -}}
  {{- end -}}
  {{- end -}}
  {{- else }}
    - http:
        paths:
  {{- if len ($.Values.ingress.customPaths) }}
  {{- "\n" }}{{ tpl (toYaml $.Values.ingress.customPaths | nindent 10) $ }}
  {{- else }}
  {{- range $p := $paths }}
          - path: {{ $p }}
            pathType: ImplementationSpecific
            backend:
              service:
                name: {{ $serviceName }}
                port:
                  number: {{ $servicePort }}
  {{- end -}}
  {{- end -}}
  {{- end -}}
Enter fullscreen mode Exit fullscreen mode

Here is a simple approach to model this ingress where the default rule is being added as would be the case in the original values.yaml configuration:

echo '    ingress:
      dashboard:
        enabled: false
        annotations: |- 
          _HT!{
            {{ if not (index . "$").Values.hull.config.specific.protocolHttp }}
            "nginx.ingress.kubernetes.io/backend-protocol": "HTTPS",
            "service.alpha.kubernetes.io/app-protocols": "{\"https\":\"HTTPS\"}"
            {{ end }}
          }
        rules:
          default:
            http:
              paths:
                root:
                  path: /
                  pathType: ImplementationSpecific
                  backend:
                    service:
                      name: dashboard
                      port:
                        number: _HT*hull.config.specific.externalPort' >> values.yaml
Enter fullscreen mode Exit fullscreen mode

If you need to change it at deploy time you could:

  • add a host to the default path
  • alter the path for the default rule for example to match your wanted route
  • set the rule for default: null in which it will not be rendered anymore and create your own rule

As a sanity check you should enable the ingress to see it prints out as expected:

echo 'hull:
  objects:
    ingress:
      dashboard:
        enabled: true' > ../configs/enable-ingress.yaml \
&& helm template -f ../configs/disable-default-rbac.yaml -f ../configs/enable-ingress.yaml .
Enter fullscreen mode Exit fullscreen mode

and there it is:

---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
kind: Service
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: dashboard
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
    kubernetes.io/cluster-service: "true"
  name: release-name-kubernetes-dashboard-dashboard
spec:
  ports:
  - name: https
    port: 443
    targetPort: https
  selector:
    app.kubernetes.io/component: dashboard
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/name: kubernetes-dashboard
  type: ClusterIP
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
    service.alpha.kubernetes.io/app-protocols: '{"https":"HTTPS"}'
  labels:
    app.kubernetes.io/component: dashboard
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-dashboard
spec:
  rules:
  - host:
    http:
      paths:
      - backend:
          service:
            name: release-name-kubernetes-dashboard-dashboard
            port:
              number: 443
        path: /
        pathType: ImplementationSpecific
  tls: []
Enter fullscreen mode Exit fullscreen mode

Since the protocolHttp option has a significant effect on the rendered output also do a quick test if everything is in order:

echo 'hull:
  config:
    specific:
      protocolHttp: true
  objects:
    ingress:
      dashboard:
        enabled: true' > ../configs/enable-ingress-http.yaml \
&& helm template -f ../configs/disable-default-rbac.yaml -f ../configs/enable-ingress-http.yaml .
Enter fullscreen mode Exit fullscreen mode

and there it is:

---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: v1
kind: Service
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: dashboard
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
    kubernetes.io/cluster-service: "true"
  name: release-name-kubernetes-dashboard-dashboard
spec:
  ports:
  - name: http
    port: 443
    targetPort: http
  selector:
    app.kubernetes.io/component: dashboard
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/name: kubernetes-dashboard
  type: ClusterIP
---
# Source: kubernetes-dashboard/templates/hull.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations: {}
  labels:
    app.kubernetes.io/component: dashboard
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubernetes-dashboard
    app.kubernetes.io/part-of: undefined
    app.kubernetes.io/version: 2.5.0
    helm.sh/chart: kubernetes-dashboard-5.2.0
  name: release-name-kubernetes-dashboard-dashboard
spec:
  rules:
  - host:
    http:
      paths:
      - backend:
          service:
            name: release-name-kubernetes-dashboard-dashboard
            port:
              number: 443
        path: /
        pathType: ImplementationSpecific
  tls: []
Enter fullscreen mode Exit fullscreen mode

Another Wrap Up

So to finish this quickly:

  1. That is how your values.yamlshould look right now:

    metrics-server:
  enabled: false
hull:
  config:
    specific:
      externalPort: 443
      protocolHttp: false
      rbac:
        clusterReadOnlyRole: false
        clusterRoleMetrics: true
      settings: {}
      pinnedCRDs: {}
  objects:
    service:
      dashboard:
        labels:
          ## Enable or disable the kubernetes.io/cluster-service label. Should be disabled for GKE clusters >=1.15.
          ## Otherwise, the addon manager will presume ownership of the service and try to delete it.
          "kubernetes.io/cluster-service": "true"
        type: ClusterIP
        ports:
          http:
            enabled: _HT?(index . "$").Values.hull.config.specific.protocolHttp
            port: _HT*hull.config.specific.externalPort
            targetPort: http
          https:
            enabled: _HT?(not (index . "$").Values.hull.config.specific.protocolHttp)
            port: _HT*hull.config.specific.externalPort
            targetPort: https
    ingress:
      dashboard:
        enabled: false
        annotations: |-
          _HT!{
            {{ if not (index . "$").Values.hull.config.specific.protocolHttp }}
            "nginx.ingress.kubernetes.io/backend-protocol": "HTTPS",
            "service.alpha.kubernetes.io/app-protocols": "{\"https\":\"HTTPS\"}"
            {{ end }}
          }
        rules:
          default:
            http:
              paths:
                root:
                  path: /
                  pathType: ImplementationSpecific
                  backend:
                    service:
                      name: dashboard
                      port:
                        number: _HT*hull.config.specific.externalPort

  2. Run the values.yaml backup:

    cp values.yaml values.tutorial-part.yaml

  3. Add in the already created objects:

    sed '1,/objects:/d' values.full.yaml > _tmp && cp values.yaml values.full.yaml && cat _tmp >> values.full.yaml && rm _tmp

Thanks for taking part and hope to see you in the next tutorial part on advanced object configuration as well!

💖 💪 🙅 🚩
gre9ory
gre9ory

Posted on May 31, 2022

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

I am purposefully super nice to AI because I imagine they are trained on data where nice people get better help. Doing this, though, has an existential uneasiness to it.
ai I am purposefully super nice to AI because I imagine they are trained on data where nice people get better help. Doing this, though, has an existential uneasiness to it.

November 28, 2024

The best way to get better at writing code is...
development The best way to get better at writing code is...

November 28, 2024

If you're a beginner, definitely check this open source guide. I've explained almost everything you need to know.
undefined If you're a beginner, definitely check this open source guide. I've explained almost everything you need to know.

November 28, 2024

HMPL has a new logo. What do you think?
webdev HMPL has a new logo. What do you think?

November 28, 2024

AI Innovations at Microsoft Ignite 2024 What You Need to Know (Part 2)
githubcopilot AI Innovations at Microsoft Ignite 2024 What You Need to Know (Part 2)

November 29, 2024