Spring & Ribbon In Kubernetes

Hi, at this moment I was interest to write something about deploying Springboot based microservices to Kubernetes environment.

So the idea is that I want to have sets of microservices spring boot based, and deployed to kubernetes as the picture bellow:

Basically we are going to have 2 microservices

The client can call these methods
-/users/v1/save --> Create User & Account
-/users/v1/all --> Get Users data
-/account/v1/all --> Get Accounts data
-/account/v1/save --> Create Account data

While user created, user_services will also call account_service to create the account

for doing this. the problems will be lay on

how user_services can call to account_services while account > service can be deployed on multiple instance and sets.
how can we make N Replicas of account_services and
User_Services can understand which account_services to
consumed
as the image bellow:

that is where the The discovery client and The Ribbon Client Load Balancing come into play.

#1 The YAML

Here is the Dependencies you should to take a look.
These are the dependencies for spring cloud kubernetes.
all the microservices must be able to be observe by other services by making the microservices discoverable by client.
to do so you'll need bellow dependencies

<dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-openfeign</artifactId>
            <version>4.0.1</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-kubernetes-ribbon</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-kubernetes-all</artifactId>
        </dependency>

also you need this dependencies, for later on used by the ingress for liveProb and healthProb mechanism.

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>

#2 The Discovery Client & Feign Clients

now, we need to make our microservices discoverable by other microservices. with spring cloud after we put the dependencies above, we can do this in our application.java

@SpringBootApplication
@EnableFeignClients
@EnableDiscoveryClient
public class GIDUsersApplicationServices {

    public static void main(String[] args) {
        SpringApplication.run(GIDUsersApplicationServices.class, args);
    }

    @Bean
    public TimedAspect timedAspect(MeterRegistry registry) {
        return new TimedAspect(registry);
    }

}

#3 The Business Logic

remember in our case we will create account during user creation on user_services. here are the code logic

@Override
    public Resources<UserDTO> saveUser(User requestData) {
        Resources<UserDTO> serviceResult = new Resources<UserDTO>();
        serviceResult.throwResponseWithCode("99", "failed to save user", null);

        UserModel userData = this.userDAO.addUser(requestData.getUserName());

        if (userData != null) {
            AccountCreation requestAccountCreation = new AccountCreation();
            requestAccountCreation.setUserId(userData.getUserId());
            requestAccountCreation.setAmount("10000");

            // ## Call request data to account services
            System.out.println("request data" + requestAccountCreation.toString());
            Resources<AccountDTO> createAccountResult = this.serviceProxy.saveUser(requestAccountCreation);
            System.out.println("response data" + createAccountResult.toString());
            // ## Call request data to account services

            if (!createAccountResult.getResponseCode().equals(ApplicationConstant.GENERAL_SUCCESS_CODE)) {
                return serviceResult.throwResponseWithCode("99", "Failed to save user", null);
            }
            UserDTO user = new UserDTO();
            user.setUser(userData);

            return serviceResult.throwSucceedResponse(user);
        } else {
            UserDTO responseOBject = new UserDTO();
            responseOBject.setErrorModel(new ErrorModel("ini error"));
            return serviceResult.throwResponseWithCode("99", "Failed to save user", responseOBject);
        }
    }

now lets zoom carefully into this line
this.serviceProxy.saveAccount

#4 The FeignClient for saveAccount

now lets take a look into the saveAccount Method bellow:

// @FeignClient(value = "account-services", url = "http://34.128.121.224:81/account")
@FeignClient(name = "gid-account-service")
public interface IAccountServicesProxy {

    @RequestMapping(method = RequestMethod.POST, value = "/account/v1/save")
    public Resources<AccountDTO> saveAccount(AccountCreation requestData);

}

as you can see, instead of using
@FeignClient(value = "account-services", url = "http://34.128.121.224:81/account") to point out the account services by its ipAddress

we can use this
@FeignClient(name = "gid-account-service") and get the available ipAddress from kubernetes Services.

as mention here in spring cloud, that

This project provides an implementation of Discovery Client for Kubernetes. This allows you to query Kubernetes endpoints (see services) by name. A service is typically exposed by the Kubernetes API server as a collection of endpoints which represent http, https addresses that a client can access from a Spring Boot application running as a pod

now we fetch all the pods ipAddress from the kubernetes services so we don't need to to put the ip address in our code anymore and let the services get the destination ip from kubernetes services. this discovery concept is similar with usage of eureka

So basically we are no longer need eureka when we use kubernetes.

Notes : in order to make this possible you have to make sure that the services name in here

spring.application.name=gid-account-service same as services
name you put on your k8s-deployment

IMPORTANT : Makesure your spring.application.name equal with your Kuberenetes Services Name

apiVersion: apps/v1
kind: Deployment # Kubernetes resource kind we are creating
metadata:
  name: gid-account-service # Required to be the same as spring config services name
  namespace: default

and you also need to make sure that you add this on your application properties, in order to enable the services become discoverable and the client load balancing on.

spring.cloud.kubernetes.discovery.enabled=true
spring.cloud.kubernetes.loadbalancer.mode=service

now we all sets. move to the deployment.

#5 The Deployment to Kubernetes.

before we go to the deployment process. make sure that you have

PostgreeDB Instance Ready to use. since the services need database
GKE Clusters
Install gcloud components
Connect to your Clusters

now here is the k8s-deployment for user_services

apiVersion: apps/v1
kind: Deployment # Kubernetes resource kind we are creating
metadata:
  name: gid-user-service
  namespace: bcad-testing-env
spec:
  selector:
    matchLabels:
      app: gid-user-service
  replicas: 2 # Number of replicas that will be created for this deployment
  template:
    metadata:
      labels:
        app: gid-user-service
    spec:
      containers:
        - name: gid-user-service
          image: noveriojoee/user_services:v0.0.6-staging.8 # Image that will be used to containers in the cluster
          imagePullPolicy: IfNotPresent
          # Configure the readiness probe
          livenessProbe:
            initialDelaySeconds: 90
            timeoutSeconds: 30
            periodSeconds: 30
            successThreshold: 1
            failureThreshold: 3
            httpGet:
              path: /actuator/health/liveness
              port: 8201
          readinessProbe:
            initialDelaySeconds: 60
            timeoutSeconds: 30
            periodSeconds: 30
            successThreshold: 1
            failureThreshold: 3
            httpGet:
              path: /actuator/health/readiness
              port: 8201
          ports:
            - containerPort: 8201 # The port that the container is running on in the cluster


---
#THIS IS SERVICES AS CONSUME BY INGRESS
apiVersion: v1
kind: Service
metadata:
  name: gid-user-service
  namespace: bcad-testing-env
  labels:
    app: gid-user-service
spec:
  selector:
    app: gid-user-service
  ports:
    - port: 8207
      protocol: TCP
      targetPort: 8201
  type: NodePort

and here is the k8s-deployment for account_services

apiVersion: apps/v1
kind: Deployment # Kubernetes resource kind we are creating
metadata:
  name: gid-account-service # Required to be the same as spring config services name
  namespace: default
spec:
  selector:
    matchLabels:
      app: gid-account-service
  replicas: 3 # Number of replicas that will be created for this deployment
  template:
    metadata:
      labels:
        app: gid-account-service
    spec:
      containers:
        - name: gid-account-services-pods
          image: noveriojoee/account_services:v0.0.3-staging.5 # Image that will be used to containers in the cluster
          imagePullPolicy: IfNotPresent
          # Configure the readiness probe
          livenessProbe:
            initialDelaySeconds: 90
            timeoutSeconds: 30
            periodSeconds: 30
            successThreshold: 1
            failureThreshold: 3
            httpGet:
              path: /actuator/health/liveness
              port: 8202
          readinessProbe:
            initialDelaySeconds: 60
            timeoutSeconds: 30
            periodSeconds: 30
            successThreshold: 1
            failureThreshold: 3
            httpGet:
              path: /actuator/health/readiness
              port: 8202
          ports:
            - containerPort: 8202 # The port that the container is running on in the cluster


---
# #THIS IS SERVICES AS CONSUME BY INGRESS
apiVersion: v1
kind: Service
metadata:
  name: gid-account-service
  namespace: default
  labels:
    app: gid-account-service
spec:
  selector:
    app: gid-account-service
  ports:
    - port: 8206
      protocol: TCP
      targetPort: 8202
  type: NodePort

by this deployment script we will have 2 sets of user_services and 3 sets of account_services

now lets exposed this to public via Ingress as our api gateway
and use GCE as the ingress controller.
read here for GCE vs NGINX Controller

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gid-user-services-ingress
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "gce" #REF -> https://medium.com/@glen.yu/nginx-ingress-or-gke-ingress-d87dd9db504c
    kubernetes.io/ingress.allow-http: "true"
    servers.com/ingress.load-balancer-healthcheck-path: "/actuator/health"
    #kubernetes.io/ingress.global-static-ip-name: gid-public-ip #REF -> Create PUBLIC STATIC IP AND IMPLEMENT TO YOUR INGRESS USING GCLOUDCONTROLLER
spec:
  # defaultBackend:
  #   service:
  #     name: gid-user-service
  #     port:
  #       number: 8207
  rules:
  - http:
      paths:
      - path: /* #<-- can be anything. considered as default
        pathType: ImplementationSpecific
        backend:
          service:
            name: gid-user-service
            port:
              number: 8207
      - path: /account/* # <-- Follow your controller request path
        pathType: ImplementationSpecific
        backend:
          service:
            name: gid-account-service
            port:
              number: 8206

by above deployment scripts now we have this.

#6 Permission

in order to ribbon be able to fetch api list ip from the namespaces you can use RBAC or you can just give the admin permission with this command

kubectl create clusterrolebinding default-pod --clusterrole cluster-admin --serviceaccount=<namespace>:default

#7. The testing

after we deploy the services and ingress now lets try to hit the api through ingress.

first

lets try Kubectl get ing -A it will show all of your ingress

now you can start calling the api with public ip 34.107.131.195

and if you take a look deeper on your ingress description using kubectl describe ing gid-user-services-ingress

you'll see that default request will be routed to user_services and all request on [ip_address]/account will be route to account_services.

Happy Trying :D

References :

Blog