EKS Security Checklist: 10 Best Practices for a Secure Cluster
CAST AI
Posted on October 21, 2022
Kubernetes is riddled with security challenges. Inevitably, the same goes for managed Kubernetes services like Amazon Elastic Kubernetes Service (EKS). The best way to tighten your cluster security is to implement practices that have become industry standard and are recommended by the Kubernetes community. Here are 10 most EKS security tactics every team needs to protect its clusters.
What exactly is Amazon EKS security about?
Amazon EKS is one of the most popular managed Kubernetes services out there. It allows teams to orchestrate their containers via Kubernetes without having to install and manage the Kubernetes control plane or the infrastructure required for Kubernetes clusters to run.
Since EKS is a service delivered by AWS, we can start the conversation about security with the shared responsibility model. In general, AWS is responsible for managing the security of its cloud services, while the customer oversees the security of the workloads.
AWS is in charge of managing the Kubernetes dashboard and control plane via EKS, including all the infrastructure services used by the cloud provider to deliver a safe Kubernetes environment.
Self-managed workers and EKS cluster configurations like IAM, pod, runtime, and network security, worker node scalability, and container image components all fall to the customer’s plate. Client-side security includes data security, upgrades and patches for worker nodes, and secure configuration for everything, starting with the data plane and nodes and ending with containers and operating systems. Customers also need to configure security groups allowing the EKS control plane to communicate with virtual private clouds (VPCs) in a secure manner.
AWS users get support from the cloud provider around EKS security in the form of:
AWS built EKS security features into its managed Kubernetes service
Here are a few built-in EKS security features every AWS user gets if they decide to run their clusters using this managed Kubernetes service.
AWS Secrets Manager
In Kubernetes, you can create key/value pairs and bring them to applications running inside your pods. If they contain any sensitive data, you can use Secret Store, which is a Container Storage Interface driver implemented by AWS Secrets Manager (as well as AWS Parameter Store).
AWS Secrets Manager gives you a central place for storing and managing Kubernetes secrets. The AWS Secrets and Configuration Provider (ASCP) plugin allows users to deal with legacy Kubernetes workloads that received secrets via ETCD. You can also apply IAM policies to define which pods are authorized to access which secret.
Identity and Access Management
Identity and Access Management (IAM) is a helping hand to every admin looking to control access to AWS resources. IAM administrators can set who can sign in and have permissions to EKS resources based on specific policies.
Users get credentials for authentication and authorization of Kubernetes resources. The idea is to grant service users access only to the features they need to perform their job.
Logging and monitoring
AWS offers access to CloudWatch, which logs stores diagnostic and audit logs from the control plane. Every EKS control plane gets its log group. It’s key that you monitor these logs to discover security and production issues on time. There’s also AWS CloudTrail, which records all the EKS activity and captures API calls made by users, roles, AWS services, or EKS console requests.
10 best practices for EKS security
1. Isolate Kubernetes nodes
Avoid exposing your Kubernetes nodes directly to public networks. Ideally, such nodes should be located on a separate network and have no direct connection to the general corporate network if that’s possible.
How to make that work? Keep Kubernetes control and data traffic isolated. Otherwise, they will both end up flowing through the same pipe. Open access to the data plane implies open access to the control plane, and that’s bad news for EKS security. Configure nodes using an ingress controller and set it to only let in connections from the master node through a specified port in the network access control list (ACL).
2. Strengthen Authentication and Authorization
Integrating Kubernetes with a third-party authentication provider is a smart move. That way, you get an extra layer of security features - for example, multi-factor authentication.
For secure control plane access, don’t manage users at the API server level but use the AWS Identity and Access Management (IAM) solution. Pick OpenID Connect (OIDC) alongside an SSO provider you got used to if you can’t get CSP IAM.1
3. Take advantage of Kubernetes Role-Based Access Control (RBAC)
Another access-related best practice for EKS security is using RBAC to define who has access to the Kubernetes API and what permissions they have. In Kubernetes 1.6 and higher, RBAC usually comes enabled by default. Given that Kubernetes combines authorization controllers, disable the legacy Attribute Based Access Control (ABAC) when turning RBAC on.
It’s a good idea to choose namespace-specific permissions over cluster-wide permissions. Even when debugging, avoid granting cluster administrator privileges. Otherwise, your container security might become compromised.
4. Avoid keeping secrets in an environment variable
Make sure that your objects use a secret in an environment variable because other parts of your system can easily access environment variables. Use secrets as files or benefit from secretKeyRef to minimize potential threats.
5. Don’t run containers in privileged mode
If your deployment has containers running in privileged (root) mode, it allows the container to gain access to important host resources. This may lead to security issues. Avoid running containers in privileged mode or turn the podSecurityPolicy on and set the privileged parameter to false. This will ensure that containers aren’t able to run processes that require root privileges on the host.
6. Don’t share the host's IPC or network namespace
Take a look at your pods and see if they share the host's IPC or network namespace. Sharing namespaces for pod and host interprocess communication is dangerous because it may open access to shared information. For this reason, pods should never be allowed to access the host namespaces.
Sharing pod and host network namespaces allows network access to the host network from the pod. This breaks network isolation. Set the hostNetwork parameter to false in PodSecurityPolicy and sleep better at night knowing that your cluster is protected.
7. Disable NET_RAW
If your Kubernetes containers don't drop the NET_RAW capability, you might enable a wide range of networking exploits from within the cluster. To ensure EKS security, use Policy Enforcement solutions like Open Policy Agents, Kyverno, or Kubernetes Pod Security admission controller to follow best industry practices. Set drop for ALL or NET_RAW capabilities in the pod's securityContext definition to make sure that the NET_RAW capability is disabled. [2, 3]
8. Check unsafe /proc mount
Deployments with unsafe /proc mount (procMount=Unmasked) allow others to bypass the default masking behavior of the container runtime. If you set a container with Unmasked /proc mount type, you may expose the host information to the container, which might result in a potential data leak or container escape. Set procMount=Default to make sure that the container doesn't expose any parts of /proc.
9. Don’t use the root file system for container security
If your containers are running without a read-only root file system, prepare for trouble. By using a read-only file system, you can prevent all kinds of malicious binaries from writing to a system or a system takeover by attackers. Make sure that containers use only the read-only file system and set the readOnlyRootFilesystem to true in Pod securityContext definition.
10. Build a rolling update strategy
Finally, to keep your EKS security tight, develop a rolling update strategy. Rolling updates let deployment updates minimize your application downtime by incrementally updating pod instances with new ones. Check out this page in the Kubernetes docs for more details.
Another point is having a vulnerability scan at runtime as there are supply chain attacks, so you need to see what realy got to your cluster even if you scanned your deployment artifacts in the CI/CD stage. In general, agent-based security solutions are good or even better then "agentless" ones.
Achieve EKS security with expert help
The Kubernetes ecosystem is evolving, and its security is no exception. As new threats emerge and problems come to light, engineers are forced to keep up with a lot of things - which takes a lot of time and effort.
Another challenge they experience around EKs security is the prioritization of security issues - depending on the application's size, prioritizing may become very time-consuming. Automation can accelerate this, winning engineers’ time back for other tasks.
CAST AI's Cloud Security feature checks clusters against industry best practices, Kubernetes recommendations, and container security system benchmarks - and then prioritizes them automatically to set you on the right track from the start.
Scan your Kubernetes cluster against configuration and security best practices, and find out how to secure optimally.
Check your Kubernetes cluster against configuration and security best practices
Connect your cluster and see your costs in 5 min, no credit card required.
References
Posted on October 21, 2022
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related
