This is a [cross-post](https://blog.bitforger.dev/installing-rancher-on-k3s-without-public-internet-access) of my blog @ [blog.bitforger.dev](https://blog.bitforger.dev)... ## What is this guide? This is for those wanting to setup Rancher on your K3s cluster that is running on a network where users from the public internet can't access it. In my instance, that means running my cluster on a separate subnet of my home network. I opted to keep this cluster internal only because I don't run anything I would want to access outside my home at the moment. If you are okay with opening holes in your firewall, enabling port forwarding, etc. to get you cluster accessible to the outside world then this guide may not be for you. Here is a quick guide to setting up [Rancher](https://rancher.com/docs/rancher/v2.6/en/) on the same cluster as your [K3s](https://rancher.com/docs/k3s/latest/en/) installation. Before we begin, I want to quickly shoutout the guides from [NetworkChuck](https://www.youtube.com/watch?v=X9fSMGkjtug) and GitHub User [kopwei](https://gist.github.com/kopwei/47dfd853261f36943aee80cc7fa5e1aa) that I used as a starting point. ## Setting Up K3s ### 1 - Set up your SD Cards - Install PiOS lite using the [Raspberry Pi Imager](https://www.raspberrypi.org/downloads.../) ![CleanShot 2022-01-01 at 01.48.26.gif](https://cdn.hashnode.com/res/hashnode/image/upload/v1641019734551/a2w8XMygr.gif) ### 2 - Init your PiOS - Put SD cards in your RPi - Turn On - Wait ~5-10 minutes - Turn Off - Take out SD cards ### 3 - Enable the needed settings. Take your initialized RPi SD cards and for each one, insert into your reader and edit the following files on the boot partition. > Replace the stuff in angle brackets with your values #### `cmdline.txt` ``` cgroup_memory=1 cgroup_enable=memory ip=:::::eth0:off ``` #### `config.txt` In the `all` key of this config file add the arm64 flag. It should look like the below. ``` [all] arm_64bit=1 ``` ### 4 - Put SD card back into your RPi and boot up. I've had the hostname not change despite changing it with the line in the `cmdline.txt` file. If that happens you can edit it by using the `sudo raspi-config` tool. ## Enable Legacy IP Tables The latest version of Raspberry PiOS doesn't have legacy IP tables installed. You can install it by running ```bash sudo apt install iptables ``` Once you have that installed you can run this: ```bash sudo iptables -F; sudo update-alternatives --set iptables /usr/sbin/iptables-legacy; sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy; sudo reboot ``` ## Install K3s Enter sudo mode ```bash sudo su - ``` Run on your master node ```bash curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE="644" sh -s ``` Get your access token by following the instruction in the output of your master node install step. Run this to set up your worker nodes ```bash curl -sfL https://get.k3s.io | K3S_TOKEN="" K3S_URL="https://:6443" K3S_NODE_NAME="" sh - ``` On the master node, run `kubectl get nodes` to check for your node status and watch them join. W00t! You should have your k3s installed and running. First off, I recommend getting your kubectl config object and loading it into your `~/.kube/config` so you can run `helm` and `kubectl` commands from your local machine. You can do that by copying your `/etc/rancher/k3s/k3s.yaml` and editing the `server` key in the file to be the IP or address as accessed from your local machine. It should probably be what you're using to SSH into your master node. Example below ```yaml apiVersion: v1 clusters: - cluster: certificate-authority-data: server: https://127.0.0.1:6443 <-- EDIT THIS name: default contexts: - context: cluster: default user: default name: default current-context: default kind: Config preferences: {} users: - name: default user: client-certificate-data: client-key-data: ``` ## Install Cert-Manager Follow the [Cert-Manager](https://cert-manager.io/docs/installation/helm/) install instructions for in-depth instructions. I'm just going to share what I did. Add helm repo ```bash helm repo add jetstack https://charts.jetstack.io helm repo update ``` Install ```bash helm install cert-manager jetstack/cert-manager \ --namespace cert-manager \ --create-namespace \ --version v1.6.1 \ --set installCRDs=true ``` Use `kubectl get pods -n cert-manager` to verify the pods are running before moving on ## Install Rancher Again, you can follow the [instructions](https://rancher.com/docs/rancher/v2.6/en/installation/install-rancher-on-k8s/) on the Rancher website if you want more in-depth discussion on what is going on. I'm just going to share what I did. Add helm repo ```bash helm repo add rancher-stable https://releases.rancher.com/server-charts/stable helm repo update ``` Create namespace ```bash kubectl create ns cattle-system ``` Install ```bash helm install rancher rancher-stable/rancher \ --namespace cattle-system \ --set hostname= \ --set bootstrapPassword=admin \ --set ingress.tls.source=letsEncrypt \ --set letsEncrypt.email= \ --set rancherImageTag=v2.6.2-linux-arm64 ``` From here, this is where you have to start playing with things... If you have [kubens](https://github.com/ahmetb/kubectx) I'd recommend switching context now. If not, you'll want to make sure you're adding the `-n cattle-system` flag to your `kubectl` commands in the rest of this post. ```bash kubens cattle-system ``` Check for an issuer ```bash kubectl get issuer ``` You should see ![CleanShot 2022-01-01 at 02.31.28@2x.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1641022319041/brqispw2P.png) You'll want to edit that Issuer to use the `dns01` challenge. Here is an example of what I did for the DigitalOcean networking. ### Create secret in the namespace Secret file ```yaml apiVersion: v1 kind: Secret metadata: name: digitalocean-dns <- This can be whatever you want to name it data: access-token: "" ``` Save the above into a file and then create it on the cluster ```bash kubectl create -f ``` Edit Issuer I used `kubectl edit issuer rancher` ```yaml apiVersion: v1 items: - apiVersion: cert-manager.io/v1 kind: Issuer metadata: annotations: meta.helm.sh/release-name: rancher meta.helm.sh/release-namespace: cattle-system creationTimestamp: "2021-12-31T03:56:49Z" generation: 2 labels: app: rancher app.kubernetes.io/managed-by: Helm chart: rancher-2.6.3 heritage: Helm release: rancher name: rancher namespace: cattle-system resourceVersion: "45795" uid: spec: acme: email: [redacted] preferredChain: "" privateKeySecretRef: name: letsencrypt-production server: https://acme-v02.api.letsencrypt.org/directory solvers: - dns01: <== REMOVE http01 and replace with dns01 solver digitalocean: tokenSecretRef: key: access-token name: digitalocean-dns ``` From there, you should only need to find the name of the certificate using `kubectl get certificate` and then delete it using `kubectl delete certificate `. Once you delete it, cert-manager will create a new certificate and certificate request and you'll be able to access your rancher portal by using the hostname you used the path `/dashboard/?setup=` ## Conclusion So, what did we do? We edited the rancher certificate issuer to use an [ACME DNS01](https://cert-manager.io/docs/configuration/acme/dns01/) solver instead of the default [HTTP01](https://cert-manager.io/docs/configuration/acme/http01/) solver. Rancher _only_ runs with HTTPS mode so we have to find a way to prove to Lets Encrypt that we control the domain we claim to control. I have DigitalOcean configured to handle the external DNS for my domain. Because of that, I was able to pass an access token to Cert Manager so they can create the TXT records needed to prove I own the domain. You should be able to use any of the setup method described in the DNS01 section of the ACME Cert-Manager docs.

bitforger

Installing Rancher on K3s without Public Internet Access

Join Our Newsletter. No Spam, Only the good stuff.