Connecting Multiple GKE Clusters to a Single Cloud SQL Instance

In a cloud-native environment, managing multiple Google Kubernetes Engine (GKE) clusters is common, especially when different parts of an application need to be isolated. However, these isolated clusters might still need to access a central database like Google Cloud SQL (PostgreSQL). In this blog post, we'll guide you through the process of connecting multiple GKE VPC-native clusters to a single Cloud SQL instance, even when they reside in separate VPC networks.

The Challenge
Imagine you have several GKE clusters, each operating in its own Virtual Private Cloud (VPC) network, for reasons of security and isolation. On the other hand, you have a Cloud SQL instance (PostgreSQL) that serves as the central repository for your application's data. The challenge is to enable seamless connectivity between any of the GKE clusters and the shared Cloud SQL instance, maintaining both network isolation and data security.

The Solution
To address this challenge, we will outline the steps to establish a secure and efficient connection between your GKE VPC-native clusters and the Cloud SQL database, regardless of the VPC network they are in.

Dedicated Node Pools
Begin by creating dedicated node pools within each GKE cluster. These node pools are optimized for backend workloads and ensure that your applications requiring database access have the necessary resources.
Provisioning Cloud SQL
Set up a PostgreSQL database instance in Google Cloud SQL. This instance will serve as the central database for your application's data.

We have many options for creating this instance: we can use Infrastructure as Code (IAC)/manual steps or gcloud command steps provided below:



gcloud sql instances create postgres-instance \
    --project=monirul-test \
    --database-version=POSTGRES_13 \
    --cpu=2 \
    --memory=4GiB \
    --root-password="DummyPass765#" \
    --availability-type=zonal \
    --zone=us-west2-a

Configuring Service Accounts After creating the service account, you need to assign the Cloud SQL Client role to it. To allow this service account to authenticate and access Cloud SQL, you need to generate a JSON key file.

Procedure to creating the service-account-key Secret from a file:



kubectl create secret generic service-account-key --from-file=key.json=/usr/local/monirul-test-c7a15664f347.json

This command will generate a key file named key.json in your current directory. This key file will be used by the Cloud SQL Proxy and your GKE application to authenticate.

Deploying the Cloud SQL Proxy

Deploying the Cloud SQL Proxy as a sidecar is a crucial step in connecting your GKE application to a Cloud SQL database securely. Below is a Kubernetes Deployment YAML manifest demonstrating how to deploy the Cloud SQL Proxy container as a sidecar alongside your main application container:



apiVersion: apps/v1
kind: Deployment
metadata:
    name: test-app-deployment
    annotations:
        linkerd.io/inject: enabled
spec:
    replicas: 1
    strategy:
        type: RollingUpdate
        rollingUpdate:
            maxSurge: 50%
            maxUnavailable: 0
    selector:
        matchLabels:
            app: test-app
    template:
        metadata:
            labels:
                app: test-app
        spec:
            affinity:
                nodeAffinity:
                    requiredDuringSchedulingIgnoredDuringExecution:
                        nodeSelectorTerms:
                        - matchExpressions:
                            - key: dedicated
                              operator: In
                              values:
                              - backend
            tolerations:
                - effect: NoSchedule
                  key: dedicated
                  operator: Equal
                  value: backend
            containers:
                - name: test-app
                  image: asia.gcr.io/monirul-test/test-app-deployment:test-app-1.0.0
                  ports:
                      - containerPort: 5000
                        name: test-app
                  resources:
                      limits:
                          cpu: '1'
                          memory: '3Gi'
                      requests:
                          cpu: '0.5'
                          memory: '2Gi'
                  env:
                      - name: DB_PASSWORD
                        valueFrom:
                            secretKeyRef:
                                name: test-app-secret
                                key: DB_PASSWORD
                      - name: APP_PROJECT_ID
                        valueFrom:
                            configMapKeyRef:
                                name: test-app-config
                                key: APP_PROJECT_ID
                      - name: DB_HOST
                        valueFrom:
                            configMapKeyRef:
                                name: test-app-config
                                key: DB_HOST
                      - name: DB_USER
                        valueFrom:
                            configMapKeyRef:
                                name: test-app-config
                                key: DB_USER
                      - name: APP_PORT
                        valueFrom:
                            configMapKeyRef:
                                name: test-app-config
                                key: APP_PORT
                      - name: DB_NAME
                        valueFrom:
                            configMapKeyRef:
                                name: test-app-config
                                key: DB_NAME
                      - name: DB_PORT
                        valueFrom:
                            configMapKeyRef:
                                name: test-app-config
                                key: DB_PORT
                      - name: DATABASE_MAX_CONNECTIONS
                        valueFrom:
                            configMapKeyRef:
                                name: test-app-config
                                key: DATABASE_MAX_CONNECTIONS
                - name: cloud-sql-proxy
                  image: gcr.io/cloudsql-docker/gce-proxy:1.22.0
                  command:
                      - '/cloud_sql_proxy'
                      - '-ip_address_types=PUBLIC'
                      - '-instances=monirul-test:us-west2:postgres-instance=tcp:0.0.0.0:5432'
                      - '-credential_file=/var/secrets/cloud-sql/key.json'
                  resources:
                      requests:
                          cpu: '100m'
                          memory: '128Mi'
                      limits:
                          cpu: '200m'
                          memory: '256Mi'
                  volumeMounts:  # Mount the Secret as a volume
                      - name: service-account-key
                        mountPath: /var/secrets/cloud-sql
            volumes:  # Define the volume that references the Secret
                - name: service-account-key
                  secret:
                      secretName: service-account-key

This YAML manifest defines a Deployment for your application ("test-app-deployment") and includes a sidecar container named "cloud-sql-proxy." The Cloud SQL Proxy handles authentication and encryption for secure connections between your GKE application and the Cloud SQL database.

Ensure you replace the image with your actual image. Once deployed, this configuration allows your GKE application to securely access the Cloud SQL database using the Cloud SQL Proxy.

ConfigMap:



apiVersion: v1
kind: ConfigMap
metadata:
    name: test-app-config
data:
    APP_PORT: '5000'
    APP_PROJECT_ID: monirul-test
    DB_NAME: postgres
    DB_HOST: localhost
    DB_USER: postgres
    DATABASE_MAX_CONNECTIONS: '15'
    DB_PORT: '5432'

Secret:



apiVersion: v1
data:
  DB_PASSWORD: RXhDZXJlODc0Iw==
kind: Secret
metadata:
  name: test-app-secret
  namespace: test-app
type: Opaque

Service.yaml



apiVersion: v1
kind: Service
metadata:
  name: test-app-svc
spec:
  ports:
    - name: test-app-svc-port
      protocol: TCP
      port: 5000
      targetPort: 5000

The provided configurations are clear and should work for your use case. Just make sure that the actual values you use for your application match the ones you've configured in the ConfigMap and Secret.

Testing and Validation Certainly, testing and validation are essential steps to ensure the GKE to Cloud SQL connection works correctly. Your provided commands for testing and validation look good:



$ k exec -it test-app-deployment-786d4bdd56-2sktd -- sh
Defaulted container "test-app" out of: test-app, cloud-sql-proxy
# curl http://localhost:5000
{"message":"hello world"}
# curl http://localhost:5000/data
[["+60102098121","Monirul","Islam","devops.monirul@gmail.com"]]

These commands will help you test various aspects of your setup, including basic connectivity and data retrieval. Be sure to conduct thorough testing, including edge cases and error scenarios, to ensure the reliability and performance of your GKE to Cloud SQL connection.

The Benefits
Connecting multiple GKE VPC-native clusters to a centralized Cloud SQL instance provides several advantages:

Isolation: Each GKE cluster remains isolated within its own VPC network, enhancing security and separation of workloads.

Centralized Data Management: Data consistency is ensured with a single Cloud SQL database instance, simplifying data management.

Scalability: This architecture can scale horizontally to handle increased workloads and data storage requirements.

Security: Strong access controls, encryption, and secure connections guarantee data confidentiality and integrity.

Efficiency: The Cloud SQL Proxy streamlines database connections, reducing latency and ensuring reliable access.

Conclusion
Connecting multiple GKE VPC-native clusters to a shared Cloud SQL instance is a critical step in building scalable, secure, and efficient cloud-native applications. With the right architecture, tools, and best practices, you can seamlessly manage your data while maintaining network isolation and data security.

Reference:
https://cloud.google.com/sql/docs/mysql/connect-kubernetes-engine

Blog

Connecting Multiple GKE Clusters to a Single Cloud SQL Instance

M.M.Monirul Islam

Join Our Newsletter. No Spam, Only the good stuff.

Related