[header image credit: Iron in the Butterfly Nebula, NASA Astronomy Picture of the Day July 21 2020 (modified)]

Making HTTP requests is a core feature of modern programming, and is often one of the first things you want to do when learning a new programming language. For Java programmers there are many ways to do it - core libraries in the JDK and third-party libraries. This post will introduce you to the Java HTTP clients that I reach for. If you use other ones, that’s great! Let me know about it. In this post I’ll cover:

Core Java:

• HttpURLConnection
• HttpClient

Popular Libraries:

• ApacheHttpClient
• OkHttp
• Retrofit

I’ll use the Astronomy Picture of the Day API from the NASA APIs for the code samples, and the code is all on GitHub in a project based on Java 11.

Core Java APIs for making Java http requests

Since Java 1.1 there has been an HTTP client in the core libraries provided with the JDK. With Java 11 a new client was added. One of these might be a good choice if you are sensitive about adding extra dependencies to your project.

Java 1.1 HttpURLConnection

First of all, do we capitalize acronyms in class names or not? Make your mind up. Anyway, close your eyes and center yourself in 1997. Titanic was rocking the box office and inspiring a thousand memes, Spice Girls had a best-selling album, but the biggest news of the year was surely HttpURLConnection being added to Java 1.1. Here’s how you would use it to make a GET request to get the APOD data:

// Create a neat value object to hold the URL
URL url = new URL("https://api.nasa.gov/planetary/apod?api_key=DEMO_KEY");

// Open a connection(?) on the URL(??) and cast the response(???)
HttpURLConnection connection = (HttpURLConnection) url.openConnection();

// Now it's "open", we can set the request method, headers etc.
connection.setRequestProperty("accept", "application/json");

// This line makes the request
InputStream responseStream = connection.getInputStream();

// Manually converting the response body InputStream to APOD using Jackson
ObjectMapper mapper = new ObjectMapper();
APOD apod = mapper.readValue(responseStream, APOD.class);

// Finally we have the response
System.out.println(apod.title);

[full code on GitHub]

This seems quite verbose, and I find the order that we have to do things is confusing (why do we set headers after opening the URL?). If you need to make more complex requests with POST bodies, or custom timeouts etc then it’s all possible but I never found this API intuitive at all.

When would you use HTTPUrlConnection, then? If you are supporting clients who are using older versions of Java, and you can’t add a dependency then this might be for you. I suspect that’s only a small minority of developers, but you might see it in older codebases - for more modern approaches, read on.

Java 11 HttpClient

More than twenty years after HttpURLConnection we had Black Panther in the cinemas and a new HTTP client added to Java 11: java.net.http.HttpClient. This has a much more logical API and can handle HTTP/2, and Websockets. It also has the option to make requests synchronously or asynchronously by using the CompletableFuture API.

99 times out of 100 when I make an HTTP request I want to read the response body into my code. Libraries that make this difficult will not spark joy in me. HttpClient accepts a BodyHandler which can convert an HTTP response into a class of your choosing. There are some built-in handlers: String, byte[] for binary data, Stream<String> which splits by lines, and a few others. You can also define your own, which might be helpful as there isn’t a built-in BodyHandler for parsing JSON. I’ve written one (here) based on Jackson following an example from Java Docs. It returns a Supplier for the APOD class, so we call .get() when we need the result.

This is a synchronous request:

// create a client
var client = HttpClient.newHttpClient();

// create a request
var request = HttpRequest
    .newBuilder(URI.create("https://api.nasa.gov/planetary/apod?api_key=DEMO_KEY"))
    .header("accept", "application/json")
    .build();

// use the client to send the request
var response = client.send(request, new JsonBodyHandler<>(APOD.class));

// the response:
System.out.println(response.body().get().title);

For an asynchronous request the client and request are made in the same way, then call .sendAsync instead of .send:

// use the client to send the request
var responseFuture = client.sendAsync(request, new JsonBodyHandler<>(APOD.class));

// We can do other things here while the request is in-flight

// This blocks until the request is complete
var response = responseFuture.get();

// the response:
System.out.println(response.body().get().title);

[full code on GitHub]

Third-party Java HTTP client libraries

If the built-in clients don’t work for you, don’t worry! There are plenty of libraries you can bring into your project which will do the job.

Apache HttpClient

The Apache Software Foundation’s HTTP clients have been around for a long time. They’re widely-used and are the foundation for a lot of higher-level libraries. The history is a little confusing. The old Commons HttpClient is no longer being developed, and the new version (also called HttpClient), is under the HttpComponents project. Version 5.0 was released in early 2020, adding HTTP/2 support. The library also supports synchronous and asynchronous requests.

Overall the API is rather low-level - you are left to implement a lot for yourself. The following code calls the NASA API. It doesn’t look too hard to use but I have skipped a lot of the error handling which you would want in production code and again I had to add Jackson code to parse the JSON response. You might also want to configure a logging framework to avoid warnings on stdout (no big deal, but it does irk me a bit). Anyway here’s the code:

ObjectMapper mapper = new ObjectMapper();

try (CloseableHttpClient client = HttpClients.createDefault()) {
    HttpGet request = new HttpGet("https://api.nasa.gov/planetary/apod?api_key=DEMO_KEY");

    APOD response = client.execute(request, httpResponse ->
        mapper.readValue(httpResponse.getEntity().getContent(), APOD.class));

    System.out.println(response.title);
}

[full code on GitHub]

Apache provides several more examples for sync and async requests.

OkHttp

OkHttp is an HTTP client from Square with a lot of helpful built-in features, like automatic handling of GZIP, response caching and retries or fallback to other hosts in case of network errors as well as HTTP/2 and WebSocket support. The API is clean although there is no built-in parsing of JSON responses so again I added code to parse the JSON with Jackson:

ObjectMapper mapper = new ObjectMapper();

OkHttpClient client = new OkHttpClient();

Request request = new Request.Builder()
    .url("https://api.nasa.gov/planetary/apod?api_key=DEMO_KEY")
    .build(); // defaults to GET

Response response = client.newCall(request).execute();

APOD apod = mapper.readValue(response.body().byteStream(), APOD.class);
System.out.println(apod.title);

[full code on GitHub]

This is fine, but the real power of OkHttp is clear when you add Retrofit over the top.

Retrofit

Retrofit is another library from Square, built on top of OkHttp. Along with all the low-level features of OkHttp it adds a way to build Java classes which abstract the HTTP details and present a nice Java-friendly API.

First, we need to create an interface which declares the methods we want to call against the APOD API, with annotations defining how those correspond to HTTP requests:

public interface APODClient {
    @GET("/planetary/apod")
    @Headers("accept: application/json")
    CompletableFuture<APOD> getApod(@Query("api_key") String apiKey);
}

The return type of CompletableFuture<APOD> makes this an asynchronous client. Square provide other adapters or you could write your own. Having an interface like this helps with mocking the client for tests, which I appreciate.

After declaring the interface we ask Retrofit to create an implementation which we can use to make requests against a given base URL. It’s also helpful for integration testing to be able to switch the base URL. To generate the client, the code looks like this:

Retrofit retrofit = new Retrofit.Builder()
    .baseUrl("https://api.nasa.gov")
    .addConverterFactory(JacksonConverterFactory.create())
    .build();

APODClient apodClient = retrofit.create(APODClient.class);

CompletableFuture<APOD> response = apodClient.getApod("DEMO_KEY");

// do other stuff here while the request is in-flight

APOD apod = response.get();System.out.println(apod.title);

[full code on GitHub]

API Authentication

If there are multiple methods in our interface which all need an API key it is possible to configure that by adding an HttpInterceptor to the base OkHttpClient. The custom client can be added to the Retrofit.Builder. The code needed to create the custom client is:

private OkHttpClient clientWithApiKey(String apiKey) {
  return new OkHttpClient.Builder()
    .addInterceptor(chain -> {
      Request originalRequest = chain.request();
      HttpUrl newUrl = originalRequest.url().newBuilder()
        .addQueryParameter("api_key", apiKey)
        .build();
      Request request = originalRequest.newBuilder()
        .url(newUrl)
        .build();
      return chain.proceed(request);
    })
  .build();
}

[full code on GitHub]

I like this kind of Java API for all but the simplest cases. Building classes to represent remote APIs is a nice abstraction that plays well with dependency injection, and having Retrofit create them for you based on a customizable OkHttp client is great.

Other HTTP clients for Java

Since posting this article on Twitter I have been delighted to see people discussing which HTTP clients they use. If none of the above is quite what you want, have a look at these suggestions:

• REST Assured - an HTTP client designed for testing your REST services. Offers a fluent interface for making requests and helpful methods for making assertions about responses.
• cvurl - a wrapper for the Java 11 HttpClient which rounds off some of the sharp edges you might encounter making complex requests.
• Feign - Similar to Retrofit, Feign can build classes from annotated interfaces. Feign is highly flexible with multiple options for making and reading requests, metrics, retries and more.
• Spring RestTemplate (synchronous) and WebClient (asynchronous) clients - if you’ve used Spring for everything else in your project it could be a good idea to stick with that ecosystem. Baeldung has an article comparing them.

Summary

There are a lot of choices for HTTP clients in Java - for simple cases I would recommend the built-in java.net.http.HttpClient. For more complex use-cases or if you want to have your HTTP APIs abstracted as Java classes as part of a larger application look at Retrofit or Feign. Happy hacking, I can’t wait to see what you build!

📧 mgilliard@twilio.com

🐦 @MaximumGilliard