Thursday, January 15, 2009

HttpURLConnection connection pool

Lately i had to set up a server to server HTTP connection with up to 200 parallel open connections. After some fiddling I succeeded with a nice and small sized URLConnection solution. It' identical for HttpURLConnection and HttpsURLConnection (with SSL). At first, you need to know that URLConnection's design requires a new URLConnection instance for every request you want to make. Cast the URLConnection to HttpURLConnection to gain access to the HTTP specific properties like requestMethod and connectionTimeout:

URL url = new URL("HTTPS", server, port, serverPath);
huc = (HttpURLConnection) url.openConnection();

Now you need some configuration. These are the properties I needed (using org.apache.commons.util.Base64):

huc.setDoOutput(true);
huc.setReadTimeout(socketTimeout);
huc.setConnectTimeout(connectionTimeout);
huc.setRequestMethod("POST");
authorizationString = "Basic " + new String(Base64.encode(username + ":" +password)).getBytes("ISO-8859-1")), "ISO-8859-1");
huc.setRequestProperty ("Authorization", authorizationString);
os = huc.getOutputStream();

Write your request (it goes to the buffer):

os.write(request);
os = huc.getOutputStream();

When you call getInputStream() your request is issued to the server and you get the response stream for further processing. You need to close the InputStream to free the underlying Socket for reuse. It's internally pooled:

try {
is = huc.getInputStream();
response = readResponse(is, huc.getContentLength());
} finally {
is.close();
}

private String readResponse(InputStream is, int length) throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int b = 0;
int read = 0;
while (read < length) {
b = is.read();
baos.write(b);
}
return baos.toString("ISO-8859-1");
}

Configure your connection pool with Java system properties:

-Dhttp.maxConnections=200
-Dsun.net.http.errorstream.enableBuffering=true

2 comments:

  1. Interesting article you got here. I'd like to read a bit more about this theme. The only thing this blog misses is a few pictures of any gadgets.
    Katherine Kripke
    Phone jammer

    ReplyDelete
  2. I tried this on Sun Jdk 1.6 but its not giving me any performance improvement between 2 subsequent calls to huc.getInputStream(). I am creating a BufferedReader like -
    BufferedReader br = new BufferedReader(new InputStreamReader(huc.getInputStream()));

    I tried with 10 requests but the avg time taken by getInputStream() is not improving (~13 s), in my case http.maxConnections=200. Any idea whats happening here & how I can improve time taken by huc.getInputStream() ?

    ReplyDelete