HTTP API of java.net.http Package With Examples

HTTP Client and WebSocket APIs provide high-level client interfaces to HTTP (versions 1.1 and 2) and low-level client interfaces to WebSocket. The main types defined are namely as follows:

• HttpClient
• HttpRequest
• HttpResponse

The protocol-specific requirements are defined in the Hypertext Transfer Protocol Version 2 (HTTP/2), the Hypertext Transfer Protocol (HTTP/1.1), and The WebSocket Protocol.

In general, asynchronous tasks execute in either the thread invoking the operation, e.g. sending an HTTP request, or by the threads supplied by the client’s executor. Dependent tasks, those that are triggered by returned CompletionStages or CompletableFutures, that do not explicitly specify an executor, execute in the same default executor as that of CompletableFuture, or the invoking thread if the operation completes before the dependent task is registered.

CompletableFutures returned by this API will throw UnsupportedOperationException for their obtrudeValue and obtrudeException methods. Invoking the cancel method on a CompletableFuture returned by this API may not interrupt the underlying operation, but may be useful to complete, exceptionally, dependent stages that have not already completed.

Unless otherwise stated, null parameter values will cause methods of all classes in this package to throw NullPointerException.

1. Interface summary 

Interface summary is as follows in the tabular format below:

HTTP Components	Action performed
HttpClient.Builder	A builder of HTTP Clients.
HttpRequest.BodyPublisher	A BodyPublisher converts high-level Java objects into a flow of byte buffers suitable for sending as a request body.
HttpRequest.Builder	A builder of HTTP requests.
HttpResponse<T>	An HTTP response.
HttpResponse.BodyHandler<T>	A handler for response bodies.
HttpResponse.BodySubscriber<T>	A BodySubscriber consumes response body bytes and converts them into a higher-level Java type.
HttpResponse.PushPromiseHandler<T>	A handler for push promises.
HttpResponse.ResponseInfo	Initial response information supplied to a BodyHandler when a response is initially received and before the body is processed.

2. A WebSocket Client.

• WebSocket.Builder: A builder of WebSocket Clients.
• WebSocket.Listener: The receiving interface of WebSocket.2. Class
• Class Description
• HttpClient

3. An HTTP Client

• HttpHeaders: A read-only view of a set of HTTP headers.
• HttpRequest: An HTTP request.
• HttpRequest.BodyPublishers: Implementations of BodyPublisher that implement various useful publishers, such as publishing the request body from a String, or from a file.
• HttpResponse.BodyHandlers: Implementations of BodyHandler that implement various useful handlers, such as handling the response body as a String, or streaming the response body to a file.
• HttpResponse.BodySubscribers: Implementations of BodySubscriber that implement various useful subscribers, such as converting the response body bytes into a String, or streaming the bytes to a file.

4. Enum summary

• HttpClient.Redirect: Defines the automatic redirection policy.
• HttpClient.VersionThe HTTP protocol version.

5. Exception Summary

gHttpConnectTimeoutException	Thrown when a connection, over which an HttpRequest is intended to be sent, is not successfully established within a specified time period.
HttpTimeoutException	Thrown when a response is not received within a specified time period.
WebSocketHandshakeException	Thrown when the opening handshake has failed.

Methods:

There are 5 ways of making HTTP requests is a core feature of modern programming, and is often one of the first things you want to do when learning a new programming language. For Java programmers, there are many ways to do it — core libraries in the JDK and third-party libraries. They are listed as below:

1. Using HttpURLConnection in J2SE
2. Using HttpClient in J2SE
3. Using ApacheHttpClient third-party libraries
4. Using OkHttp third-party libraries
5. Using Retrofit third-party libraries

Let us discuss them with an illustration justifying the same.

Way 1: Core java 

Core Java APIs for making Java HTTP requests. Since Java 1.1 there has been an HTTP client in the core libraries provided with the JDK. With Java 11 a new client was added. One of these might be a good choice if you are sensitive about adding extra dependencies to your project.

Note: For codes coverage here astronomy picture of the Day API from the NASA APIs for the code samples.

1.1 Java 1.1 HttpURLConnection

First, do we capitalize acronyms in class names or not? Make your mind up. Anyway, close your eyes and center yourself in 1997. Titanic was rocking the box office and inspiring a thousand memes, Spice Girls had a best-selling album, but the biggest news of the year was surely HttpURLConnection being added to Java 1.1.  It is justified in the illustration provided as below:

Illustration: 

Usage and how to get GET request to make in order to get the APOD data

// Step 1: Create a neat value object to hold the URL
URL url = new URL("https://api.nasa.gov/planetary/apod?api_key=DEMO_KEY");

// Step 2: Open a connection(?) on the URL(??) and cast the response(???)
HttpURLConnection connection = (HttpURLConnection) url.openConnection();

// Now it's "open", we can set the request method, headers etc.
connection.setRequestProperty("accept", "application/json");

// This line makes the request
InputStream responseStream = connection.getInputStream();

// Step 3: Manually converting the response body InputStream to
// APOD using Jackson
ObjectMapper mapper = new ObjectMapper();

APOD apod = mapper.readValue(responseStream, APOD.class);

// Step 5: Finally, display the response we have
System.out.println(apod.title);

This seems quite verbose, and I find the order that we have to do things is confusing (why do we set headers after opening the URL?). If you need to make more complex requests with POST bodies, or custom timeouts, etc then it’s all possible, but I never found this API intuitive at all.

When would you use HTTPUrlConnection, then? If you are supporting clients who are using older versions of Java, and you can’t add a dependency then this might be for you. I suspect that’s only a small minority of developers, but you might see it in older codebases for more modern approaches, read on.

1.2 Java 11 HttpClient

More than twenty years after HttpURLConnection we had Black Panther in the cinemas and a new HTTP client added to Java 11: java.net.http.HttpClient. This has a much more logical API and can handle HTTP/2, and Websockets. It also has the option to make requests synchronously or asynchronously by using the CompletableFuture API.

99 times out of 100 when I make an HTTP request I want to read the response body into my code. Libraries that make this difficult will not spark joy in me. HttpClient accepts a BodyHandler which can convert an HTTP response into a class of your choosing. There are some built-in handlers: String, byte[] for binary data, Stream<String> which splits bylines, and a few others. You can also define your own, which might be helpful as there isn’t a built-in BodyHandler for parsing JSON. I’ve written one (here) based on Jackson following an example from Java Docs. It returns a Supplier for the APOD class, so we call .get() when we need the result.

Illustration: Synchronous request

// Step 1: Create a client
var client = HttpClient.newHttpClient();

// Step 2: Create a request
var request = HttpRequest.newBuilder(URI.create("https://api.nasa.gov/planetary/apod?api_key=DEMO_KEY")).header("accept", "application/json").build();

// Step 3: Now use the client to send the request

var response = client.send(request, new JsonBodyHandler<>(APOD.class));


// Response
System.out.println(response.body().get().title);

// For an asynchronous request the client and request are made
// in the same way

// Step 3:  After this call .sendAsync instead of .send:
// Step 4: Use the client to send the request
var responseFuture = client.sendAsync(request, new JsonBodyHandler<>(APOD.class));

// We can do other things here while the request is in-flight
// This blocks until the request is complete
var response = responseFuture.get();

// Response
System.out.println(response.body().get().title);

Way 2: Third-party Java HTTP client libraries. If the built-in clients don’t work for you, don’t worry! There are plenty of libraries you can bring into your project which will do the job.

2.1 Library 1: Apache HttpClient

The Apache Software Foundation’s HTTP clients have been around for a long time. They’re widely used and are the foundation for a lot of higher-level libraries. The history is a little confusing. The old Commons HttpClient is no longer being developed, and the new version (also called HttpClient), is under the HttpComponents project. Version 5.0 was released in early 2020, adding HTTP/2 support. The library also supports synchronous and asynchronous requests.

Overall the API is rather low-level, you are left to implement a lot for yourself. The following code calls the NASA API. It doesn’t look too hard to use, but I have skipped a lot of the error handling which you would want in production code, and again I had to add Jackson code to parse the JSON response. You might also want to configure a logging framework to avoid warnings on stdout (no big deal, but it does irk me a bit).

Apache provides several more examples for sync and async requests.

Illustration:

ObjectMapper mapper = new ObjectMapper();

try (CloseableHttpClient client = HttpClients.createDefault()) 
{

  HttpGet request = 
  new HttpGet("https://api.nasa.gov/planetary/apod?api_key=DEMO_KEY");

  APOD response = client.execute(request, httpResponse -> 
  mapper.readValue(httpResponse.getEntity().getContent(), APOD.class));

  System.out.println(response.title);
}

2.2 Library 2: OkHttp

OkHttp is an HTTP client from Square with a lot of helpful built-in features, like automatic handling of GZIP, response caching and retries or fallback to other hosts in case of network errors as well as HTTP/2 and WebSocket support. The API is clean, although there is no built-in parsing of JSON responses.

Illustration: Parsing of JSON with Jackson

ObjectMapper mapper = new ObjectMapper();
OkHttpClient client = new OkHttpClient();

Request request = new Request.Builder().url("https://api.nasa.gov/planetary/apod?api_key=DEMO_KEY").build(); // defaults to GET

Response response = client.newCall(request).execute();

APOD apod = mapper.readValue(response.body().byteStream(), APOD.class);

System.out.println(apod.title);

Note: This is fine, but the real power of OkHttp is clear when you add Retrofit over the top.

2.3 Library 3: Retrofit

Retrofit is another library from Square, built on top of OkHttp. Along with all the low-level features of OkHttp, it adds a way to build Java classes that abstract the HTTP details and present a nice Java-friendly API.

2.3.1 First, we need to create an interface that declares the methods we want to call against the APOD API, with annotations defining how those correspond to HTTP requests which is as follows:

public interface APODClient 
{
  @GET("/planetary/apod")
  @Headers("accept: application/json")
  CompletableFuture<APOD> getApod(@Query("api_key") String apiKey);

}

2.3.2 The return type of CompletableFuture<APOD> makes this an asynchronous client. Square provides other adapters, or you could write your own. Having an interface like this helps with mocking the client for tests, which is appreciated.

2.3.3 After declaring the interface we ask Retrofit to create an implementation that we can use to make requests against a given base URL. It’s also helpful for integration testing to be able to switch the base URL.

Illustration: Generating the client 

Retrofit retrofit = new Retrofit.Builder().baseUrl("https://api.nasa.gov").addConverterFactory(JacksonConverterFactory.create()).build();

APODClient apodClient = retrofit.create(APODClient.class);

CompletableFuture<APOD> response = apodClient.getApod("DEMO_KEY");

// Do all other stuffs here
// while the request is in-flight

APOD apod = response.get();

System.out.println(apod.title);

API Authentication

If there are multiple methods in our interface which all need an API key it is possible to configure that by adding an HttpInterceptor to the base OkHttpClient. The custom client can be added to the Retrofit.Builder.  The sample code is as follows:

Implementation: Creating the custom client 

private OkHttpClient clientWithApiKey(String apiKey) {
    return new OkHttpClient.Builder()

    .addInterceptor(chain -> {

        Request originalRequest = chain.request();

        HttpUrl newUrl = originalRequest.url().newBuilder()

        .addQueryParameter("api_key", apiKey).build();

        Request request = originalRequest.newBuilder().url(newUrl).build();

        return chain.proceed(request);

    }).build();

}

Here this kind of java API is preferred for all but the simplest cases. Building classes to represent remote APIs is a nice abstraction that plays well with dependency injection, and having Retrofit create them for you based on a customizable OkHttp client is great.

Other HTTP clients for Java 

If none of the above is quite what you want, have a look at the suggestions been listed below:

• REST Assured, an HTTP client designed for testing your REST services. Offers a fluent interface for making requests and helpful methods for making assertions about responses.
• cvurl is a wrapper for the Java 11 HttpClient which rounds off some sharp edges you might encounter making complex requests.
• Feign – Similar to Retrofit, Feign can build classes from annotated interfaces. Feign is highly flexible with multiple options for making and reading requests, metrics, retries, and more.
• Spring RestTemplate (synchronous) and WebClient (asynchronous) clients – if you’ve used Spring for everything else in your project it could be a good idea to stick with that ecosystem. Baeldung has an article comparing them.
• MicroProfile Rest Client – another client in the “build a class from an annotated interface” mode, this one is interesting because you can reuse the same interface to create a webserver too, and be sure that the client and server match. If you’re building service and a client for that service then it could be the one for you.

A. Password authentication

I like this kind of Java API for all but the simplest cases. Building classes to represent remote APIs is a nice abstraction that plays well with dependency injection, and having Retrofit create them for you based on a customizable OkHttp client is great. Here we can use the PasswordAuthentication class which is just a holder of these values.

Note: Every request should use the same username and password. The Authenticator class provides a number of getXXX (for example, getRequestingSite()) methods that can be used to find out what values should be provided.

Example 

import java.io.*;

class {

    public static void main (String[] args) {

        HttpClient.newBuilder().authenticator(new Authenticator() {

            // @Override
            protectedPasswordAuthenticationgetPasswordAuthentication() {

                return new PasswordAuthentication( "username", "password".toCharArray());
            }

        }).build();
    }
}

B. Setting the redirect policy

When calling a website page, sometimes the page you want to access has been moved to a different address. In this case, you’ll receive HTTP status code 3xx, usually with the information about the new URI. By setting an appropriate redirect policy, HttpClient can redirect the request to the new URI automatically. All redirect policies are defined and described in enum with the name HttpClient.Redirect.

Note: Using the followRedirects() method, you can set the redirect policy

HttpClient.newBuilder()
followRedirects(HttpClient.Redirect.ALWAYS)
build();

C. Sending Sync or Asynchronous requests

• HttpClient provides two possibilities for sending a request to a server:
• send(…) synchronously (blocks until the response comes)
• sendAsync(…) asynchronously (doesn’t wait for a response, non-blocking)
• Up until now, the send(…) method naturally waits for a response:

Example 1:

HttpResponse<String> response = HttpClient.newBuilder()
.build()
.send(request, BodyHandlers.ofString());

This call returns an HttpResponse object and this means that the next instruction from your application flow will be executed only when the response is already returned

This approach has a lot of drawbacks, especially when you are processing large amounts of data. To overcome this limitation, you can use the sendAsync(…) method, which returns a CompletableFeature<String> to process a request asynchronously:

CompletableFuture<String> response = HttpClient.newBuilder()
.build()
.sendAsync(request, BodyHandlers.ofString())
.thenApply(HttpResponse::body);

Note: The API can also deal with multiple responses and stream the request and response bodies

Example 2:

class  {

    public static void main (String[] args) {
        
        List<URI>uris = Arrays.asList()

new URI("https://postman-echo.com/get?foo1=bar1"),
new URI("https://postman-echo.com/get?foo2=bar2");


HttpClient client = HttpClient.newHttpClient();


List<HttpRequest> requests = uris.stream()
.map(HttpRequest::newBuilder)
.map(reqBuilder ->reqBuilder.build())
.collect(Collectors.toList());



CompletableFuture.allOf(requests.stream()
.map(request ->client.sendAsync(request, ofString()))
.toArray(CompletableFuture<?>[]::new))
.join();


    }
}

CompletableFuture.allOf(requests.stream()
.map(request ->client.sendAsync(request, ofString()))
.toArray(CompletableFuture<?>[]::new))
.join();
   }
}

Code explanation and interlinking are as follows:

So your main code will keep on executing, configuring the callback in the future, and thenAccept. But this callback will only be triggered once the server returns a response. The HTTP client will use a background thread to make the call. Be aware, the server response will take a while. And in the meantime, your application will have ended. So what do you do to make this example work? Call the join method in the future. This joins the application thread that your code runs on with the future. At this point in the code, the join method will wait until the future has been completed. And if it’s completed, that also means that your thenAccept callback will run. And indeed, when you run this example, you get back the expected results.

Example 3:

class {

    // Main driver method
    public static void main (String[] args) {

        HttpClient.newBuilder().authenticator(new Authenticator() {

            // @Override
            protectedPasswordAuthenticationgetPasswordAuthentication() {

                return new PasswordAuthentication("username", "password".toCharArray());
            }

        }).build();

    }
}

CompletableFuture<HttpResponse<String>> response1 = HttpClient.newBuilder()
.executor(executor)
.build()
.sendAsync(request,asString());

Note: By default, the HttpClient uses executor java.util.concurrent.Executors.newCachedThreadPool().

The synchronous and blocking send API is easier to use, but the asynchronous API will help you create responsive and more scalable applications, so you’ll have to choose what best fits your use case.

Goals for the HTTP Client API

The HTTP Client APIs has many goals designed to help you understand the important characteristics of this API and how you can use it in your programming:

• Easy to use for common cases, including a simple blocking mode.
• Simple and concise API that caters to 80-90 percent of application needs
• Supports standard and common authentication mechanisms
• Easy to set up the WebSocket interface handshake
• Friendly to embedded-system requirements; in particular, the avoidance of permanently running timer threads

Note: Supports HTTPS/TLS

• It must be performance oriented and its memory footprint less than older and third-party APIs
• Provides non-blocking request and response semantics through CompletableFuture, which can be chained to trigger dependent actions.