Groovy and HTTP

Kelly RobinsonFebruary 12th, 2013Last Updated: February 11th, 2013

1 1,888 7 minutes read

This article originally appeared in the December 2012 issue of GroovyMag.

Some different ways that Groovy makes interacting with the web easier

One of the major benefits of Groovy is how it simplifies some of the common scenarios we deal with in Java. Complex code with conditionals, error handling and many other concerns can be expressed in a very concise and easily understandable fashion. This article will touch on some convenient Groovy-isms related to interacting with content over HTTP. First we’ll look at some of the syntactic sugar added to the standard Java classes that simplify GET and POST requests, and then we’ll take a look at how the

HTTPBuilder module provides a DSL for using the HttpClient library.

The test project

In order to provide an environment for putting up a website and demonstrating various HTTP requests, we’ll be using the Gradle Jetty plugin and some simple Groovlets. The full source code is available at https://github.com/kellyrob99/groovy-http and I hope you’ll clone a copy to take a closer look. The simple index page contains the ‘hello world’ content shown in Listing 1.

<!DOCTYPE html>
<html>
<head>
    <title>Groovy HTTP</title>
</head>
<body>
<p>hello world</p>
</body>
</html>

Listing 1: Our ‘hello world’ index page used for testing

We’ll start with the simplest available methods for interacting with HTTP using Groovy and no additional library support.

Groovy methods added to String and URL

The DefaultGroovyMethods class provides a couple of very handy methods to enhance the default operation of the String and URL classes. In particular for String we have a new toURL() method and, for URL, the text property. In addition, the URL class is enhanced with convenience methods for working with associated InputStream and OutputStreams.

String.toURL()

This is a small gain as all you’re really doing is avoiding a call to new URL(String spec). The difference in keystrokes isn’t large but, combined with some other MetaClass benefits of Groovy, it can be very helpful for creating fluent and easily understandable code.

URL.text()

This seemingly small addition to the API of the URL class abstracts away a lot of the usual boilerplate involved in streaming content over a URLConnection. Underneath the hood is a very sensible implementation that buffers the underlying connection and automatically handles the closing of all resources for you. For most use cases the default behaviour is likely to be sufficient but, if not, there are overloaded URL.text(String charset) and URL.text(Map parameters, String charset) methods that allow for modification and handle more specifics of the connection configuration.
The one line invocation in Listing 2 demonstrates how to load an html page, returning the raw html as a String.

String html = 'http://localhost:8081/groovy-http'.toURL().text

Listing 2: One liner to initiate an HTTP GET request for an html page

There’s still a lot that could go wrong using this shorthand syntax for an HTTP request, as several exceptions might be thrown depending on whether or not the url is correctly formatted, or if the content specified doesn’t exist. The Spock test shown in Listing 3 exercises both of these conditions. Note that a 404 response will result in a FileNotFoundException.

@Unroll('The url #url should throw an exception of type #exception')
def 'exceptions can be thrown converting a String to URL and accessing the text'() {
    when:
    String html = url.toURL().text

    then:
    def e = thrown(exception)

    where:
    url                          | exception
    'htp://foo.com'              | MalformedURLException
    'http://google.com/notThere' | FileNotFoundException
}

Listing 3: Spock test showing some possible failure conditions for our GET request

For comparison let’s take a look at what the same GET request looks like using a URL in Java, shown in Listing 4.

URL html = new URL('http://localhost:8081/groovy-http/index.html');
URLConnection urlConnection = html.openConnection();
BufferedReader reader = new BufferedReader(
	new InputStreamReader(urlConnection.getInputStream()));
StringBuffer response = new StringBuffer();
String inputLine;
while ((inputLine = reader.readLine()) != null)
{
	response.append(inputLine)
}
reader.close();

Listing 4: The Java version of reading from a URLConnection (based on the canonical example from Oracle.com)

There’s still no error handling in place in the Java version which is obviously a much more verbose way to load the same data.

POST with URL streams

Similarly to simplifying GET requests, executing a POST using Groovy can take advantage of some of the enhancements to common Java classes. In particular, simplified stream handling allows for tight, correct and expressive coding. Listing 5 shows a Spock test configuring the URLConnection, POSTing some data and reading back the result from the connection.

private static final String POST_RESPONSE = 'Successfully posted [arg:[foo]] with method POST'    

def 'POST from a URLConnection'() {
    when:
    final HttpURLConnection connection = makeURL('post.groovy').toURL().openConnection()
    connection.setDoOutput(true)
    connection.outputStream.withWriter { Writer writer ->
        writer << 'arg=foo'
    }

    String response = connection.inputStream.withReader { Reader reader -> reader.text }

    then:
    connection.responseCode == HttpServletResponse.SC_OK
    response == POST_RESPONSE
}

Listing 5: POST request using Groovy and a URLConnection

Notice that we don’t have to explicitly cast the connection to HttpUrlConnection in order to get the responseCode back, and that we don’t have to explicitly close any of the streams used. Also, we don’t need to create local variables for the Reader/Writer object as we would have to in Java; similarly no calls to ‘new’ are required, as Object creation is all hidden behind the convenience methods. The equivalent Java code requires four calls to new and two to close(), as well as much more involved code for extracting the result. The canonical example of how to do this in Java can be seen on http://docs.oracle.com/javase/tutorial/networking/urls/readingWriting.html

Note that you can also parse response content very easily using the XmlSlurper / XmlParser and JsonSlurper classes included in the standard Groovy distribution.

HttpClient and HTTPBuilder make things even easier

The reality is that in most modern Java applications developers have some nice alternatives to directly working with URL and URLConnection objects for working with HTTP. One of the more popular libraries available is HttpClient and its successor HttpComponents. Wrappers for all of the HTTP verbs are provided which simplifies configuration, execution and consumption of responses. Listing 6 shows a Spock test using HttpClient and mirroring our prior GET examples.

def 'HttpClient example in Java'() {
    when:
    HttpClient httpclient = new DefaultHttpClient();
    HttpGet httpget = new HttpGet(makeURL('helloWorld.groovy'));
    ResponseHandler<String> responseHandler = new BasicResponseHandler();
    String responseBody = httpclient.execute(httpget, responseHandler);

    then:
    responseBody == HELLO_WORLD_HTML
}

Listing 6: HttpClient GET example

This can be further reduced if there is no need for keeping the intermediate variables around. In fact, we can get it down to the single line shown in Listing 7.

String response = new DefaultHttpClient().execute(new HttpGet(makeURL('helloWorld.groovy')), new BasicResponseHandler())

Listing 7: HttpClient GET one-liner

This is obviously a lot easier on the eyes and very clear in intent. The HttpClient library also has convenience mechanisms for declaring common behaviour across connections, an API for providing custom response parsing implementations and automatic handling for (most of) the underlying resource streams and connections. For those of us using Groovy, there’s a nice wrapper for HttpClient called HTTPBuilder that adds a DSL-style configuration mechanism and some very nice features in terms of error handling and content parsing. Listing 8 shows our standard GET example again, this time working against an object called http assigned from new HTTPBuilder(Object uri). Note that we’re using Groovy’s multiple assignment feature to return and assign multiple values from our Closure.

def 'GET with HTTPBuilder'() {
    when:
    def (html, responseStatus) = http.get(path: 'helloWorld.groovy', contentType: TEXT) { resp, reader ->
        [reader.text, resp.status]
    }

    then:
    responseStatus == HttpServletResponse.SC_OK
    html == HELLO_WORLD_HTML
}

Listing 8: Spock test showing Groovy HTTPBuilder GET support

If you noticed in Listing 8 I explicitly set the request with contentType: TEXT, it’s because HTTPBuilder by default provides automatic response content type detection and parsing. Since I’m requesting an xml document, HTTPBuilder can automatically parse the result with Groovy’s XmlSlurper. HTTPBuilder can also detect that it is an html page and pass the response through NekoHTML first to ensure that you’re working with a well-formed document. Listing 9 shows the slight difference in how we could interact with the parsed response content and the reader in our Closure from Listing 8 is quietly replaced with a GPathResult referring to the parsed content.

def 'GET with HTTPBuilder and automatic parsing'() {
    when:
    def (html, responseStatus) = http.get(path: 'helloWorld.groovy') { resp, reader ->
        [reader, resp.status]
    }

    then:
    responseStatus == HttpServletResponse.SC_OK
    html instanceof GPathResult
    html.BODY.P.text() == 'hello world'
}

Listing 9: automatic detection and parsing of xml

It’s unlikely that you’re going to be parsing a lot of html this way but with the abundance of xml services available nowadays automated parsing can be very helpful. The same applies for JSON and if we give a hint as to the contentType we can get back a parsed JSONObject when interacting with such services as shown in Listing 10.

def 'GET with HTTPBuilder and automatic JSON parsing'() {
    when:
    def (json, responseStatus) = http.get(path: 'indexJson.groovy', contentType: JSON) { resp, reader ->
        [reader, resp.status]
    }

    then:
    responseStatus == HttpServletResponse.SC_OK
    json instanceof JSONObject
    json.html.body.p == 'hello world'
}

Listing 10: automatic parsing of JSON responses

The HTTPBuilder module also has some convenience methods for handling failure conditions. By allowing for specifying both default failure handlers and specific behaviour for individual requests you’ve got lots of options at your disposal. Listing 11 shows how to define a default failure handler that simply traps the response code. Note that the Closure used for hading GET response is never run since in this case the page we’re requesting results in an HTTP 404 Not Found response code.

def 'GET with HTTPBuilder and error handling'() {
    when:
    int responseStatus
    http.handler.failure = { resp ->
        responseStatus = resp.status
    }
    http.get(path: 'notThere.groovy', contentType: TEXT) { resp, reader ->
        throw new IllegalStateException('should not be executed')
    }

    then:
    responseStatus == HttpServletResponse.SC_NOT_FOUND
}

Listing 11: Defining a failure handler with HTTPBuilder

POSTing data with HTTPBuilder is also very straightforward, requiring only an additional body parameter as shown in Listing 12.

def 'POST with HTTPBuilder'() {
    when:
    def (response, responseStatus) = http.post(path: 'post.groovy', body: [arg: 'foo']) { resp, reader ->
        [reader.text(),resp.status]
    }

    then:
    responseStatus == HttpServletResponse.SC_OK
    response == POST_RESPONSE
}

Listing 12: POST using HTTPBuilder

HTTPBuilder also provides some more specific abstractions for dealing with certain scenarios. There’s RESTClient for dealing with RESTful webservices in a simplified manner, there’s AsyncHTTPBuilder for asynchronously executing requests and for the Google App Engine, which doesn’t allow socket based connections, there’s the HttpURLClient which wraps HttpUrlConnection usage.

Conclusion

Hopefully this has given you a taste for what Groovy can do to help you with HTTP interactions and gives you some ideas for making your own HTTP client applications a bit Groovier.

Groovy and HTTP

Some different ways that Groovy makes interacting with the web easier

The test project

Listing 1: Our ‘hello world’ index page used for testing

Groovy methods added to String and URL

String.toURL()

URL.text()

Listing 2: One liner to initiate an HTTP GET request for an html page

Listing 3: Spock test showing some possible failure conditions for our GET request

Listing 4: The Java version of reading from a URLConnection (based on the canonical example from Oracle.com)

POST with URL streams

Listing 5: POST request using Groovy and a URLConnection

HttpClient and HTTPBuilder make things even easier

Listing 6: HttpClient GET example

Listing 7: HttpClient GET one-liner

Listing 8: Spock test showing Groovy HTTPBuilder GET support

Listing 9: automatic detection and parsing of xml

Listing 10: automatic parsing of JSON responses

Listing 11: Defining a failure handler with HTTPBuilder

Listing 12: POST using HTTPBuilder

Conclusion

More reading

Thank you!

Kelly Robinson

Thank you!

Some different ways that Groovy makes interacting with the web easier

The test project

Listing 1: Our ‘hello world’ index page used for testing

Groovy methods added to String and URL

String.toURL()

URL.text()

Listing 2: One liner to initiate an HTTP GET request for an html page

Listing 3: Spock test showing some possible failure conditions for our GET request

Listing 4: The Java version of reading from a URLConnection (based on the canonical example from Oracle.com)

POST with URL streams

Listing 5: POST request using Groovy and a URLConnection

HttpClient and HTTPBuilder make things even easier

Listing 6: HttpClient GET example

Listing 7: HttpClient GET one-liner

Listing 8: Spock test showing Groovy HTTPBuilder GET support

Listing 9: automatic detection and parsing of xml

Listing 10: automatic parsing of JSON responses

Listing 11: Defining a failure handler with HTTPBuilder

Listing 12: POST using HTTPBuilder

Conclusion

More reading

Thank you!

Related Articles

Thank you!