Client Side Load Balancing External REST Calls with Java and Spring Boot

Targets

Java 8+, Spring Boot 5, Any Client Side Framework such as Angular/React/Vue/etc.

Introduction

As uncommon of a need as it may be, there may come a time where you are required to write an application that makes various calls to an external REST API / microservice that exists in multiple locations for the purpose of high availability. Because there are multiple instances of the service running, it just doesn't seem right to configure your application with a proxy that only hits one of the multiple endpoints, thus making calls to the service in a load balanced manner might be your ultimate goal.

There are likely better ways of load balancing service calls in the enterprise, such as through load balancing hardware, or an implementation of a GSLB (Global Server Load Balancer) product such as NGINX. However with that being said, the need may arise where your application is required to perform this task without the extra expenses and/or additional potential points of failure.

If you are not familiar with setting up a Spring Boot application, and/or want more background into where I found my inspiration, I strongly encourage you to go through the Client-Side Load-Balancing with Spring Cloud LoadBalancer guide, before continuing on.

When a Proxy Isn't Enough

As a secondary need, when it comes to consuming an external API from a browser based application, is the Cross-Origin Resource Sharing (CORS) issue which occurs when a browser detects that a call is being made to an API that exists on a different domain. This can be solved by including Access-Control-Allow-Origin in the response header of the service; but, for the sake of this exercise, we are going to suppose that we either cannot, or do not want, to make this change within the service code.

With Spring Boot, one can use the Netflix Zuul Proxy to get around this issue, which is extraordinarily easy to configure. Unfortunately, you are going to be limited to consuming a single configured endpoint. If you do some more research, you will undoubtedly come across the Netflix Ribbon, which has much more promise in getting us to where we need to be, but is substantially more complex to setup than a Zuul Proxy, and even more disappointing, you will come to find out that Ribbon has officially been deprecated.

Spring Cloud LoadBalancer

Enter Spring Cloud LoadBalancer, which picks up where the Netflix Ribbon left off, and in fact it actually uses Ribbon under the hood. The official Spring site has some good information on how one can use LoadBalancer in the aforementioned guide: Client-Side Load-Balancing with Spring Cloud LoadBalancer, from which I base our solution.

Spring's guide comes pretty close, but doesn't really address our exact need. That is, what we ultimately need to be able to do: Intercept all API calls being made from our client application running in Spring Boot (gets, posts, puts, and deletes), and redirect them under the hood, in a load balanced manner, to several different external endpoints, without having to define each specific API call.

Our Scenario

We have a client application (lets say Angular) with a service that communicates with a RESTful service that exists at multiple endpoints for high-availability. We do not want our application to simply be configured to hit just one of these endpoints, so we are going to use Spring Boot to take advantage of the Spring Cloud LoadBalancer. Lets say that we have 3 external endpoints which host the API(s) that we want to be able to consume in our application:

https://myservice.host1.com, https://myservice.host2.com, https://myservice.host3.com

We want our client application to call the APIs via the "/api" path, as if these endpoints existed within our app. Therefore when calls are made, such as "/api/getthis" or "/api/savethat" we will have our Java Spring Boot application, that lives in the same space as our client application, intercept and redirect those calls to one of the 3 external endpoints. The following demonstrates one way that you can set this up.

Setting up the Java Spring Boot Load Balancer

After having setup your Spring Boot application (which is easy to do here), go to your main application class, which for my example, I have named LBExampleApplication, and make sure it contains the following:

package com.lb.example;

import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.client.loadbalancer.reactive.ReactorLoadBalancerExchangeFilterFunction;
import org.springframework.http.HttpHeaders;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.reactive.function.client.WebClient;
import reactor.core.CoreSubscriber;
import reactor.core.publisher.Mono;
import javax.servlet.http.HttpServletRequest;
import java.io.IOException;
import java.util.Enumeration;
import java.util.stream.Collectors;

@Slf4j
@SpringBootApplication
@RestController
public class LBExampleApplication {
   private final WebClient.Builder loadBalancedWebClientBuilder;
   private final ReactorLoadBalancerExchangeFilterFunction lbFunction;

   public LBExampleApplication(WebClient.Builder webClientBuilder,
                           ReactorLoadBalancerExchangeFilterFunction lbFunction) {
      this.loadBalancedWebClientBuilder = webClientBuilder;
      this.lbFunction = lbFunction;
   }

   public static void main(String[] args) {
      SpringApplication.run(LBExampleApplication.class, args);
   }

   @RequestMapping(value = "/api/*")
   public Mono<String> balanceIt(HttpServletRequest request) throws IOException {
      Boolean logDebugInfo = true;
      String uri = request.getRequestURI();
      String type = request.getMethod();
      Enumeration<String> headerNames = request.getHeaderNames();
      String url = "http://load-balancer" + uri;
      String body =  request.getReader().lines().collect(Collectors.joining(System.lineSeparator()));

      if (logDebugInfo) {
         if (headerNames != null) {
            while (headerNames.hasMoreElements()) {
               log.info(" --- Request Header: " + request.getHeader(headerNames.nextElement()));
            }
         }
         log.info(" --- Request Body: " + body);
         log.info(" --- " + type + " Reroute to Load Balancer: " + url);
      }

      switch (type) {
         case "GET":
            return loadBalancedWebClientBuilder.build().get().uri(url)
                  .retrieve().bodyToMono(String.class);
         case "POST":
                return loadBalancedWebClientBuilder.build().post().uri(url)
                  .header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
                  .bodyValue(body)
                  .retrieve().bodyToMono(String.class);
         case "PUT":
            return loadBalancedWebClientBuilder.build().put().uri(url)
                  .header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
                  .bodyValue(body)
                  .retrieve().bodyToMono(String.class);
         case "DELETE":
            return loadBalancedWebClientBuilder.build().delete().uri(url)
                  .retrieve().bodyToMono(String.class);
      }

      return new Mono<String>() {
         @Override
         public void subscribe(CoreSubscriber<? super String> actual) {
         }
      };
   }
}

NOTE: Logging has been added so that you can see what is going on in the debug console when running locally.

Next, create a class called LBExampleConfiguration

package com.lb.example;

import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.client.DefaultServiceInstance;
import org.springframework.cloud.client.ServiceInstance;
import org.springframework.cloud.loadbalancer.core.ServiceInstanceListSupplier;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import reactor.core.publisher.Flux;
import javax.annotation.PostConstruct;
import java.util.ArrayList;
import java.util.List;

@Configuration
public class LBExampleConfiguration {
    @Bean
    @Primary
    ServiceInstanceListSupplier serviceInstanceListSupplier() {
        return new IMServiceInstanceListSuppler("load-balancer");
    }
}

@Slf4j
class IMServiceInstanceListSuppler implements ServiceInstanceListSupplier {
    private final String serviceId;

    @Value("#{'${api.endpoints}'.split(',')}")
    private List<String> endpoints;

    private List<ServiceInstance> instances;

    IMServiceInstanceListSuppler(String serviceId) {
        this.serviceId = serviceId;
    }

    @PostConstruct
    void buildServiceInstance(){
        instances = new ArrayList<>();
        Integer sid = 0;
        for (String host : endpoints) {
            sid++;
            instances.add(new DefaultServiceInstance(serviceId + sid, serviceId, host, 443, true));
            log.info(" --- Added Host: " + host);
        }
    }

    @Override
    public String getServiceId() {
        return serviceId;
    }

    @Override
    public Flux<List<ServiceInstance>> get() {
        return Flux.just(instances);
    }
}

Next, create a class called WebClientConfig

package com.lb.example;

import org.springframework.cloud.client.loadbalancer.LoadBalanced;
import org.springframework.cloud.loadbalancer.annotation.LoadBalancerClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.reactive.function.client.WebClient;

@Configuration
@LoadBalancerClient(name = "api-gateway", configuration = LBExampleConfiguration.class)
public class WebClientConfig {
    @LoadBalanced
    @Bean
    WebClient.Builder webClientBuilder() {
        return WebClient.builder();
    }
}

Finally, within your application.yml

api:
  endpoints: "myservice.host1.com,myservice.host2.com,myservice.host3.com"

This is a comma delimited list where you define all of the endpoints in which you want your API calls to be load balanced. Don't include the http or https here, because functionality within the LBExampleConfiguration class takes care of this based on the port number specified in the call to new DefaultServiceInstance(...)

Setting up the Client Side

Well, there actually isn't much to do here. From your client application, whether it is Angular, React, Vue, or anything else. Just call your API(s) as if they exist on your backend. Your Java Spring Boot Load Balancer is essentially a simple backend application that will handle routes directed to /api/ and direct them to their actual external endpoints.

Blog