Setup your own Kubernetes Cluster with Kops and AWS Infrastructure
Abhay
Posted on March 1, 2020
As you started reading this article, so I believe you must known What is kubernetes and AWS.
Pre-requisite configuration:
AWS Account ā https://portal.aws.amazon.com/billing/signup
Kubectl (Kubernetes Command-lin Tool):
Install Kubectl on your system ā https://kubernetes.io/docs/tasks/tools/install-kubectl/KOPS (Kubernetes Operations):
Install Kops on your system, follow this official documentation ā https://github.com/kubernetes/kops#installingInstall AWS CLI and configure - https://aws.amazon.com/cli/
Verify installation by execcuting aws --version
You need to create new user on AWS. However you can use root user, but its not recommended at all.
- Open IAM console: https://console.aws.amazon.com/iam/
- In the navigation pane, choose Users and then choose Add user
- Type user name for new user. I am using
kops
as a username for simplicity - Select type of access as Programmatic access
- Choose Next for Permission and give admin access to this user with AdministratorAccess Policy
- Choose next for Tags and Review
- This will creates an Access Key ID and Secret Access Key. Store them securely as You will not have access to the secret access key again after this step.
Use command aws configure
and enter Access Key ID, Secret Access Key and Default Region Name on prompt. I am using ap-south-1
which is Asia Pacific server at Mumbai. See list of aws regions here.
Deploying Kubernetes to AWS
https://kops.sigs.k8s.io/getting_started/aws/
Setup IAM user
Create an IAM user with following permissions:
AmazonEC2FullAccess AmazonRoute53FullAccess AmazonS3FullAccess IAMFullAccess AmazonVPCFullAccess
Do it with command line>
aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AmazonEC2FullAccess --group-name kops
aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AmazonRoute53FullAccess --group-name kops
aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess --group-name kops
aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/IAMFullAccess --group-name kops
aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AmazonVPCFullAccess --group-name kops
aws iam create-user --user-name kops
aws iam add-user-to-group --user-name kops --group-name kops
aws iam create-access-key --user-name kops
Buy a domain (If don't have already)
I have buy (:p) https://kubernetes.cf for this tutorial
Create Hosted Zone in AWS
Create hosted zone in AWS and update NS in DNS of domain provider
Test the DNS setup
dig ns dev.kubernetes.cf
should get something like
;; ANSWER SECTION:
dev.kubernetes.cf. 172800 IN NS ns-1.awsdns-1.net.
dev.kubernetes.cf. 172800 IN NS ns-2.awsdns-2.org.
dev.kubernetes.cf. 172800 IN NS ns-3.awsdns-3.com.
dev.kubernetes.cf. 172800 IN NS ns-4.awsdns-4.co.uk.
Create cluster state storage on S3
aws s3api create-bucket \
--bucket dev-kubernetes-cf-state-store \
--region ap-south-1
--create-bucket-configuration LocationConstraint=<region>
aws s3api put-bucket-versioning --bucket dev-kubernetes-cf-state-store --versioning-configuration Status=Enabled
aws s3api put-bucket-encryption --bucket dev-kubernetes-cf-state-store --server-side-encryption-configuration '{"Rules":[{"ApplyServerSideEncryptionByDefault":{"SSEAlgorithm":"AES256"}}]}'
Creating the Cluster
export NAME=dev.kubernetes.cf
export KOPS_STATE_STORE=s3://dev-kubernetes-cf-state-store
aws ec2 describe-availability-zones --region ap-south-1
- create secret
generate local keys:
ssh-keygen
kops create secret --name dev.kubernetes.cf sshpublickey admin -i ~/.ssh/id_rsa.pub
create SSL using AWS Certificate Manager
kops create cluster \
--zones ap-south-1a \
--state s3://dev-kubernetes-cf-state-store \
--api-ssl-certificate arn:aws:acm:[aws-cert-key-id} \
${NAME}
<!-- --topology private \ -->
Master and Worker nodes are configurable in termas of specification and count
If you want to edit something
kops edit cluster ${NAME}
Finally apply cluster configuration
kops update cluster ${NAME} --yes
kubectl get nodes
use following command to validate cluster is up and running, it may take up to 10-15 minutes to complete setup
kops validate cluster
Install Kubernetes Dashboard
Connect to master
ssh to the master: ssh -i ~/.ssh/id_rsa admin@api.dev.kubernetes.cf
Install Kube Dashboard
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta6/aio/deploy/recommended.yaml
Create the service account in the current namespace
kubectl create serviceaccount my-dashboard-sa
Give that service account root access on the cluster
kubectl create clusterrolebinding my-dashboard-sa \
--clusterrole=cluster-admin \
--serviceaccount=default:my-dashboard-sa`
Find the secret that was created to hold the token for the SA
kubectl get secrets
Show the contents of the secret to extract the token
kubectl describe secret my-dashboard-sa-token-xxxxx
run kubectl proxy
Install ngnix-controller with helm
Install Helm - https://helm.sh/docs/intro/install/
helm install nginx-cntroller nginx/nginx-ingress
apply SSL on Load Balancer
update load balancer endpoint url in Route53 as alias target
change loadbalancer instance port of 443 same as 80
create service i.e. goapp
User followign docker image for quick reference: https://hub.docker.com/r/abygawade/goapp
create an ingress with following configuration
apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: dev-ingress spec: rules: - host: dev.kubernetes.cf http: paths: - path: /go backend: serviceName: goapp servicePort: 80
Visit: http://dev.kubernetes.cf/go
https://dev.kubernetes.cf/go
Replace kubernetes.cf with your domain name
Posted on March 1, 2020
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.