This article is part of our Academy Course titled Developing Modern Applications with Scala.
In this course, we provide a framework and toolset so that you can develop modern Scala applications. We cover a wide range of topics, from SBT build and reactive applications, to testing and database acceess. With our straightforward tutorials, you will be able to get your own projects up and running in minimum time. Check it out here!
1. Introduction
How often these days you hear phrases like “Web APIs are eating the world”? Indeed, credits to Marc Andreessen for nicely summarizing that, but APIs become more and more important in backing business-to-business or business-to-consumer conversations, particularly in web universe.
For many businesses the presence of the web APIs is must-have competitive advantage and often is a matter of survival.
Table Of Contents
2. Being REST(ful)
In the today’s web, HTTP is the king. Over the years many different attempts have been made to come up with standard, uniform high-level protocol to expose services over it. SOAP was probably the first one which gained widespread popularity (particularly in enterprise world) and served as de-facto specification for web services and APIs development for number of years. However, its verbosity and formalism often served as a source of additional complexity and that is one of the reasons for Representational state transfer (or just REST) to emerge.
These days most of the web services and APIs are developed following REST architectural styles and as such are called REST(ful). There are myriads of great frameworks available for building web services and APIs following REST(ful) principles and constraints, but in the world of Scala, there is an unquestionable leader: Akka HTTP.
3. From Spray to Akka Http
Many of you may be familiar with Akka HTTP by means of its terrific predecessor, very popular Spray Framework. It is still quite widely used in the wild but since last year or so, Spray Framework is being actively migrated under Akka HTTP umbrella and is going to be discontinued eventually. As of moment of writing, the latest stable version of Akka HTTP (distributed as part of beloved Akka Toolkit) was 2.4.11.
Akka HTTP stands on shoulders of Actor Model (provided by Akka core) and Akka Streams and as such, fully embraces reactive programming paradigm. However, please take a note that some parts of the Akka HTTP are still wearing the experimental label as the migration of the Spray Framework to the new home is ongoing and some contracts very likely will change.
4. Staying on the Server
In general, when we are talking about web services and APIs, there are at least two parties involved: the server (provider) and the client (consumer). Unsurprisingly, Akka HTTP has the support for both so let us start from the most interesting part, the server-side.
Akka HTTP offers quite a few layers of the APIs, ranging from pretty low-level request/response processing up to the beautiful DSLs. Along this part of the tutorial, we are going to use Akka HTTP server DSL only, as it is the fastest (and prettiest) way to start building your REST(ful) web services and APIs in Scala.
4.1. Routes and Directives
In the core of the Akka HTTP servers is routing which in context of HTTP-based communications could be described as the process of selecting best paths to handle incoming requests. Routes are composed and described using directives: the building blocks of Akka HTTP server-side DSL.
For example, let us develop a simple REST(ful) web API to manage users, somewhat functionally similar to the one we have done in “Web Applications with Play Framework” section. Just to remind, here is how our User case class looks like (not surprisingly, it is the same one we have used everywhere):
case class User(id: Option[Int], email: String, firstName: Option[String], lastName: Option[String])
Probably, the first route we may need our web API to handle is to return the list of all users, so let it be our starting point:
val route: Route = path("users") {
pathEndOrSingleSlash {
get {
complete(repository.findAll)
}
}
}
As simple as that! Every time the client will issue GET request to the /users endpoint (as per path("users") and get directives), we are going to fetch all the users from underlying data storage and return them back (as per complete directive). In reality, routes could be quite complex, requiring extraction and validation of different parameters but luckily, routing DSL has all that built-in, for example:
val route: Route = pathPrefix("users") {
path(IntNumber) { id =>
get {
rejectEmptyResponse {
complete(repository.find(id))
}
}
}
}
Let us take a closer look on this code snippet. As you may already guess, we are providing a web API to retrieve a user by its integer identifier, which is supplied as URI path parameter (using path(IntNumber) directive), for example /users/101. However, the user with such identifier may or may not exist (generally speaking, repository.find(id) returns Option[User]). In this case, the presence of the rejectEmptyResponse directive instructs our endpoint to return a 404 HTTP status code (rather than empty response) in case user was not found in the data storage.
Looks really easy and concise, but curious readers may be wondered what data format will be used to represent the users?
4.2. Marshalling and Unmarshalling
Akka HTTP uses the process of marshalling and unmarshalling to convert objects into the representation, transferable over the wire. But practically, most of the time when REST(ful) web APIs concerned, we are talking about JSON format and Akka HTTP has a superior support for that.
import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._ import spray.json.DefaultJsonProtocol._ implicit val userFormat = jsonFormat4(User)
The use of predefined jsonFormat4 function introduces implicit support for marshalling User case class into JSON representation and unmarshalling it back to User from JSON, for example:
val route: Route = pathPrefix("users") {
pathEndOrSingleSlash {
post {
entity(as[User]) { user =>
complete(Created, repository.insert(user))
}
}
}
}
Looks not bad so far but REST(ful) web API practitioners may argue that semantics of POST action should include Location header to point out to the newly created resource. Let us fix that, it would require to restructure the implementation just a bit, introducing onSuccess and respondWithHeader directives.
val route: Route = pathPrefix("users") {
pathEndOrSingleSlash {
post {
entity(as[User]) { user =>
onSuccess(repository.insert(user)) { u =>
respondWithHeader(Location(uri.withPath(uri.path / u.id.mkString))) {
complete(Created, u)
}
}
}
}
}
}
Excellent, last but not least, there is a dedicated support for streaming the HTTP responses in JSON format, particularly when Akka Streams are engaged. In the “Database Access with Slick” section we have already learned how to stream the results from data store using awesome Slick library, so this code snipped should look very familiar.
class UsersRepository(val config: DatabaseConfig[JdbcProfile]) extends Db with UsersTable {
...
def stream(implicit materializer: Materializer) = Source
.fromPublisher(db.stream(
users.result.withStatementParameters(fetchSize = 10)))
...
}
The Source[T, _] could be directly returned from complete directive, assuming that T (which in our case is User) has implicit support for JSON marshalling, for example:
implicit val jsonStreamingSupport = EntityStreamingSupport.json()
val route: Route = pathPrefix("users") {
path("stream") {
get {
complete(repository.stream)
}
}
}
Although it may look no different from other code snippets we have seen so far, in this case Akka HTTP is going to use chunked transfer encoding to deliver the response.
Also, please notice that JSON support is currently in transitioning phase and still resides in the Spray Framework packages. Hopefully, the eventual Akka HTTP abstractions would be very similar and migration would be just a matter of package change (fingers crossed).
4.3. Directives in Depth
Directives in Akka HTTP are able to do mostly everything with request or response, and there is a really impressive list of predefined ones. In order to understand the mechanics better, we are going to take a look on two more examples: logging and security.
The logRequestResult directive becomes of great help if you need to troubleshoot your Akka HTTP web APIs by inspecting the complete snapshots of the requests and responses.
val route: Route = logRequestResult("user-routes") {
...
}
Here just a quick illustration of how this information may look like in the log output when a POST request is issued against /users endpoint:
[akka.actor.ActorSystemImpl(akka-http-webapi)] user-routes: Response for
Request : HttpRequest(HttpMethod(POST),http://localhost:58080/users,List(Host: localhost:58080, User-Agent: curl/7.47.1, Accept: */*, Timeout-Access: ),HttpEntity.Strict(application/json,{"email": "a@b.com"}),HttpProtocol(HTTP/1.1))
Response: Complete(HttpResponse(200 OK,List(),HttpEntity.Strict(application/json,{"id":1,"email":"a@b.com"}),HttpProtocol(HTTP/1.1)))
Please take a note that all request and response headers along with the payload are included. In case there is a sensitive information passed around (like passwords or credentials), it would be a good idea to implement some kind of filtering or masking on top.
Another interesting example is related to securing your Akka HTTP web APIs using authenticateXxx directives family. At the moment, there are only two authentication flows supported: HTTP Basic Auth and OAuth2. As an example, let us introduce yet another endpoint to our web API to allow user modification (typically done using PUT request).
val route: Route = pathPrefix("users") {
path(IntNumber) { id =>
put {
authenticateBasicAsync(realm = "Users", authenticator) { user =>
rejectEmptyResponse {
entity(as[UserUpdate]) { user =>
complete(
repository.update(id, user.firstName, user.lastName) map {
case true => repository.find(id)
case _=> Future.successful(None)
}
)
}
}
}
}
}
}
This endpoint is protected using HTTP Basic Auth and would verify if user’s email already exists in the data storage. For simplicity, the password is always hard-coded to "password" (please never do that in real applications).
def authenticator(credentials: Credentials): Future[Option[User]] = {
credentials match {
case p @ Credentials.Provided(email) if p.verify("password") =>
repository.findByEmail(email)
case _ => Future.successful(None)
}
}
Really nice, but probably the best part of the Akka HTTP design is extensibility built right into the core: it is very easy to introduce your own directives in case there is nothing predefined to satisfy your needs.
4.4. When things go wrong
For sure, most of the time your web APIs are going to work perfectly fine, servicing happy clients. However, from time to time bad things happen and it is better to be prepared to deal with them. Essentially, there could be many reasons for failure: violation of business constraints, database connectivity, unavailability of the external dependencies, garbage collection, … Under the hood we could classify them into two different buckets: exceptions and execution duration.
In light of the web API we are developing, the typical example of exceptional situation would be the creation of the user with duplicate email address. On the data storage level it would lead to unique constraint violation and needs a specific treatment. As you may expect, there is a dedicated directive for that, handleExceptions, for example:
val route: Route = pathPrefix("users") {
pathEndOrSingleSlash {
post {
handleExceptions(exceptionHandler) {
extractUri { uri =>
entity(as[User]) { user =>
onSuccess(repository.insert(user)) { u =>
complete(Created, u)
}
}
}
}
}
}
}
The handleExceptions directive accepts an ExceptionHandler as an argument which maps the particular exception to response. In our case, it would be SQLException mapped to HTTP response code 409, indicating conflict.
val exceptionHandler = ExceptionHandler {
case ex: SQLException => complete(Conflict, ex.getMessage)
}
That’s great, but what about execution time? Luckily, Akka HTTP supports a variety of different timeouts to protect your web APIs. Request timeout is one of those which allows to limit the maximum amount of time a route may take to return a response. If this timeout is exceeded, HTTP response code 503 will be returned, signaling that web API is not available at the moment. All these timeouts could be configured using global settings in the application.conf or using timeout directives, for example:
val route: Route = pathPrefix("users") {
pathEndOrSingleSlash {
post {
withRequestTimeout(5 seconds) {
...
}
}
}
}
The ability to use such a fine-grained control is a really powerful option as not all web APIs are equally important and the execution expectations may vary.
4.5. Startup / Shutdown
Akka HTTP uses Akka’s extension mechanisms and provides an extension implementation for Http support. There are quite a few ways to initialize and use it but the simplest one would be using bindAndHandle function.
implicit val system = ActorSystem("akka-http-webapi")
implicit val materializer = ActorMaterializer()
val repository = new UsersRepository(config)
Http().bindAndHandle(new UserApi(repository).route, "0.0.0.0", 58080)
The shutdown procedure is somewhat tricky but in the nutshell involves two steps: unbinding Http extension and shutting down the underlying actor system, for example:
val f = Http().bindAndHandle(new UserApi(repository).route, "0.0.0.0", 58080)
f.flatMap(_.unbind) onComplete {
_ => system.terminate
}
Probably you would rarely encounter a need to use programmatic termination however it is good to know the mechanics of that nonetheless.
4.6. Secure HTTP (HTTPS)
Along with plain HTTP, Akka HTTP is ready for production deployments and supports secure communication over HTTPS as well. Similarly to what we have learned in “Web Applications with Play Framework” section of the tutorial, we would need a Java Key Store with certificate(s) imported into it. For the development purposes, generating self-signed certificates is good enough option to start with:
keytool -genkeypair -v -alias localhost -dname "CN=localhost" -keystore src/main/resources/akka-http-webapi.jks -keypass changeme -storepass changeme -keyalg RSA -keysize 4096 -ext KeyUsage:critical="keyCertSign" -ext BasicConstraints:critical="ca:true" -validity 365
However, configuring server-side HTTPS support requires quite a bit of code to be written, luckily it is very well documented. The SslSupport trait serves as an example of such a configuration.
trait SslSupport {
val configuration = ConfigFactory.load()
val password = configuration.getString("keystore.password").toCharArray()
val https: HttpsConnectionContext =
managed(getClass.getResourceAsStream("/akka-http-webapi.jks")).map { in =>
val keyStore = KeyStore.getInstance("JKS")
keyStore.load(in, password)
val keyManagerFactory = KeyManagerFactory.getInstance("SunX509")
keyManagerFactory.init(keyStore, password)
val tmf = TrustManagerFactory.getInstance("SunX509")
tmf.init(keyStore)
val sslContext = SSLContext.getInstance("TLS")
sslContext.init(keyManagerFactory.getKeyManagers, tmf.getTrustManagers,
new SecureRandom)
ConnectionContext.https(sslContext)
}.opt.get
}
Now we can use https connection context variable to create a HTTPS binding by passing it to bindAndHandle function as an argument, for example:
Http().bindAndHandle(new UserApi(repository).route, "0.0.0.0", 58083, connectionContext = https)
Please notice that you may have HTTP support, HTTPS support or both at the same time, Akka HTTP gives you a full freedom in making these kinds of choices.
4.7. Testing
There are many different strategies of how to approach web APIs testing. It comes with no surprise that as every other Akka module, Akka HTTP has dedicated test scaffolding, seamlessly integrated with ScalaTest framework (unfortunately specs2 is not supported out of the box yet).
Let us start with very simple example to make sure that /users endpoint is going to return an empty list of users when GET request is sent.
"Users Web API" should {
"return all users" in {
Get("/users") ~> route ~> check {
responseAs[Seq[User]] shouldEqual Seq()
}
}
}
Very simple and easy, the test case looks close to perfection! And those test cases are amazingly fast to execute because they are run against route definition, without bootstrapping the full-fledged HTTP server instance.
A bit more complicated scenario would be to develop a test case for user modification, which requires user to be created before as well as HTTP Basic Authentication credentials to be passed.
"Users Web API" should {
"create and update user" in {
Post("/users", User(None, "a@b.com", None, None)) ~> route ~> check {
status shouldEqual Created
header[Location] map { location =>
val credentials = BasicHttpCredentials("a@b.com", "password")
Put(location.uri, UserUpdate(Some("John"), Some("Smith"))) ~>
addCredentials(credentials) ~> route ~> check {
status shouldEqual OK
responseAs[User] should have {
'firstName (Some("John"))
'lastName (Some("Smith"))
}
}
}
}
}
}
Certainly, if you are practicing TDD or derivative methodologies (and I truly believe everyone should), you would enjoy the robustness and conciseness of the Akka HTTP test scaffolding. It feels just done right.
5. Living as a Client
I hope at this point we truly appreciated the powerful server-side support which Akka HTTP provides for REST(ful) web services and APIs development. But often the web APIs we built themselves become the clients for other external web services and APIs.
As we already mentioned before, Akka HTTP has excellent client-side support for communicating with external HTTP-based web services and APIs. Similarly to the server-side, there are multiple levels of client APIs available, but probably the easiest one to use is request-level API.
val response: Future[Seq[User]] =
Http()
.singleRequest(HttpRequest(uri = "http://localhost:58080/users"))
.flatMap { response => Unmarshal(response).to[Seq[User]] }
In this simple example there is just a single request issues to our /users endpoint and results are unmarshalled from JSON to Seq[User]. However, in most real-world applications it is costly to pay the price of establishing HTTP connections all the time so using the connection pools is a preferred and effective solution. It is good to know that Akka HTTP also has all the necessary building blocks to cover these scenarios as well, for example:
val pool = Http().superPool[String]()
val response: Future[Seq[User]] =
Source.single(HttpRequest(uri = "http://localhost:58080/users") -> uuid)
.via(pool)
.runWith(Sink.head)
.flatMap {
case (Success(response), _) => Unmarshal(response).to[Seq[User]]
case _ => Future.successful(Seq[User]())
}
The end results of those two code snippets are exactly the same. But the latter one uses pool and shows one more time that Akka Streams are consistently supported by Akka HTTP and are quite handy in processing client-side HTTP communications. One important detail though: once finished, the connection pools should be shutdown manually, for example:
Http().shutdownAllConnectionPools()
In case the client and server communication relies on HTTPS protocol, Akka HTTP client-side API supports TLS encryption out of the box.
6. Conclusions
If you are building or thinking about building REST(ful) web services and APIs in Scala definitely give Akka HTTP a try. Its routing DSL is beautiful and elegant way of describing the semantic of your REST(ful) resources and plugging the implementation to service them. In comparison to Play Framework, there are quite a few overlaps indeed but for pure web API projects Akka HTTP is definitely a winner. The presence of some experimental modules may impose a few risks in adopting Akka HTTP right away however every release is getting closer and closer to deliver stable and concise contracts.
7. What’s next
It is really sad to admit, but our tutorial is getting to its end. But I honestly believe that for most of us it becomes just a beginning of the exciting journey into the world of Scala programming language and ecosystem. Let the joy and success be with you along the way!
The complete source code is available for download.
