Getting started with Scala and Scalatra – Part III

Jos DirksenSeptember 24th, 2012Last Updated: October 22nd, 2012

0 29 9 minutes read

This post is the third on a series of articles I’m writing on scalatra. In ‘part I’ we created the initial environment, and in ‘part II’ we created the first part of a REST API and added some tests. In this third part of the scalatra tutorial we’re going to look at the following topics:

Persistency: we use scalaquery to persist elements from our model.
Security: handle a security header containing an API key.

Frist we’ll look at the persistency part. For this part we’ll be using scalaquery. Note that the code we show here is pretty much the same for scalaquery’s successor slick. Slick, however, requires scala 2.10.0-M7 and this would mean we have to alter our complete scala setup. So for this example we’ll just use scalaquery (whose syntax is the same of slick). If you haven’t done so already, install JRebel so your changes are reflected instantly without having to restart the service.

Persistency

I’ve used postgresql for this example, but any of the databases supported by scalaquery can be used. The database model I’ve used is a very simple one:

CREATE TABLE sc_bid
(
  id integer NOT NULL DEFAULT nextval('sc_bid_id_seq1'::regclass),
  'for' integer,
  min numeric,
  max numeric,
  currency text,
  bidder integer,
  date numeric,
  CONSTRAINT sc_bid_pkey1 PRIMARY KEY (id ),
  CONSTRAINT sc_bid_bidder_fkey FOREIGN KEY (bidder)
      REFERENCES sc_user (id) MATCH SIMPLE
      ON UPDATE NO ACTION ON DELETE NO ACTION,
  CONSTRAINT sc_bid_for_fkey FOREIGN KEY ('for')
      REFERENCES sc_item (id) MATCH SIMPLE
      ON UPDATE NO ACTION ON DELETE NO ACTION
)
 
CREATE TABLE sc_item
(
  id integer NOT NULL DEFAULT nextval('sc_bid_id_seq'::regclass),
  name text,
  price numeric,
  currency text,
  description text,
  owner integer,
  CONSTRAINT sc_bid_pkey PRIMARY KEY (id ),
  CONSTRAINT sc_bid_owner_fkey FOREIGN KEY (owner)
      REFERENCES sc_user (id) MATCH SIMPLE
      ON UPDATE NO ACTION ON DELETE NO ACTION
)
 
CREATE TABLE sc_user
(
  id serial NOT NULL,
  username text,
  firstname text,
  lastname text,
  CONSTRAINT sc_user_pkey PRIMARY KEY (id )
)

As you can a simple model, with a couple of foreign keys and primary keys that are autogenerated. We define a table for the users, for the items and for the bids. Note that this is database specific so this will only work for postgresql. An additional note on postgresql and scalaquery. Scalaquery doesn’t support schemas. This means that we have to define the tables in the ‘public’ schema.

Before we can start working with scalaquery we first have to add it to our project. In the build.sbt add the following dependencies

  'org.scalaquery' %% 'scalaquery' % '0.10.0-M1',
  'postgresql' % 'postgresql' % '9.1-901.jdbc4'

After updating you’ll have the scalaquery and postgres jars you need. Lets look at one of the repositories: the bidrepository and the RepositoryBase trait.

// the trait
import org.scalaquery.session.Database
 
trait RepositoryBase {
  val db = Database.forURL('jdbc:postgresql://localhost/dutch_gis?user=jos&password=secret', driver = 'org.postgresql.Driver')
}
 
// simple implementation of the bidrepository
package org.smartjava.scalatra.repository
import org.smartjava.scalatra.model.Bid
import org.scalaquery.session._
import org.scalaquery.ql.basic.{BasicTable => Table}
import org.scalaquery.ql.TypeMapper._
import org.scalaquery.ql._
import org.scalaquery.ql.extended.PostgresDriver.Implicit._
import org.scalaquery.session.Database.threadLocalSession
 
class BidRepository extends RepositoryBase {
 
  object BidMapping extends Table[(Option[Long], Long, Double, Double, String, Long, Long)]('sc_bid') {
      def id = column[Option[Long]]('id', O PrimaryKey)
      def forItem = column[Long]('for', O NotNull)
      def min = column[Double]('min', O NotNull)
      def max = column[Double]('max', O NotNull)
      def currency = column[String]('currency')
      def bidder = column[Long]('bidder', O NotNull)
      def date = column[Long]('date', O NotNull)
 
      def noID = forItem ~ min ~ max ~ currency ~ bidder ~ date
      def * = id ~ forItem ~ min ~ max ~ currency ~ bidder ~ date
  }
 
  /**
   * Return a Option[Bid] if found or None otherwise
   */
  def get(bid: Long, user: String) : Option[Bid] = {
      var result:Option[Bid] = None;
 
      db withSession {
          // define the query and what we want as result
       val query = for (u <-BidMapping if u.id === bid) yield u.id ~ u.forItem ~ u.min ~ u.max ~ u.currency ~ u.bidder ~ u.date
 
       // map the results to a Bid object
       val inter = query mapResult {
         case(id,forItem,min,max,currency,bidder,date) => Option(new Bid(id,forItem, min, max, currency, bidder, date));
       }
 
       // check if there is one in the list and return it, or None otherwise
       result = inter.list match {
         case _ :: tail => inter.first
         case Nil => None
       }
      }
 
      // return the found bid
      result
    }
 
    /**
     * Create a bid using scala query. This will always create a new bid
     */
    def create(bid: Bid): Bid = {
      var id: Long = -1;
 
      // start a db session
      db withSession {
        // create a new bid
        val res = BidMapping.noID insert (bid.forItem.longValue, bid.minimum.doubleValue, bid.maximum.doubleValue, bid.currency, bid.bidder.toLong, System.currentTimeMillis());
        // get the autogenerated bid
        val idQuery = Query(SimpleFunction.nullary[Long]('LASTVAL'));
        id = idQuery.list().head;
      }
      // create a bid to return
      val createdBid = new Bid(Option(id), bid.forItem, bid.minimum, bid.maximum, bid.currency, bid.bidder, bid.date);
      createdBid;
    }
 
    /**
     * Delete a bid
     */
    def delete(user:String, bid: Long) : Option[Bid] = {
      // get the bid we're deleting
      val result = get(bid,user);
 
      // delete the bid
      val toDelete = BidMapping where (_.id === bid)
      db withSession {
        toDelete.delete
      }
 
      // return deleted bid
      result
    }
}

Looks complex, right? We’ll it isn’t once you’ve got the hang of how scalaquery works. With scalaquery you create a table mapping. In this mapping you specify the type of fields you expect. In this example our mapping table looks like this:

 
  object BidMapping extends Table[(Option[Long], Long, Double, Double, String, Long, Long)]('sc_bid') {
      def id = column[Option[Long]]('id', O PrimaryKey)
      def forItem = column[Long]('for', O NotNull)
      def min = column[Double]('min', O NotNull)
      def max = column[Double]('max', O NotNull)
      def currency = column[String]('currency')
      def bidder = column[Long]('bidder', O NotNull)
      def date = column[Long]('date', O NotNull)
 
      def noID = forItem ~ min ~ max ~ currency ~ bidder ~ date
      def * = id ~ forItem ~ min ~ max ~ currency ~ bidder ~ date
  }

Here we define the mapping of the table ‘sc_bid’. For each field, we define the name of the column and it’s type. If we want we can add specific options that are taken into account when you create your ddl from this (not something I’ve used for this example). The last two defs define the ‘constructors’ for this mapping. The ‘def *’ is the default constructor, where we have all the fields beforehand, the ‘def noID’ is the one we’ll use when we create a bid for this first time and we don’t have an id yet. Remember the ids are autogenerated by the database.
With this mapping we can start writing our repository functions. Lets start with the first one: get

  /**
   * Return a Option[Bid] if found or None otherwise
   */
  def get(bid: Long, user: String) : Option[Bid] = {
      var result:Option[Bid] = None;
 
      db withSession {
          // define the query and what we want as result
       val query = for (u <-BidMapping if u.id === bid) yield u.id ~ u.forItem ~ u.min ~ u.max ~ u.currency ~ u.bidder ~ u.date
 
       // map the results to a Bid object
       val inter = query mapResult {
         case(id,forItem,min,max,currency,bidder,date) => Option(new Bid(id,forItem, min, max, currency, bidder, date));
       }
 
       // check if there is one in the list and return it, or None otherwise
       result = inter.list match {
         case _ :: tail => inter.first
         case Nil => None
       }
      }
 
      // return the found bid
      result
    }

Here you can see that we use the standard scala for construct to create a query iterate over the table mapped with BidMapping. To make sure we only get the field we want we apply a filter using the ‘if u.id === bid’ statement. In the yield statement we specify the fields we want to return. By using the mapResult on the query we can process the results from the query and convert it to our case object and add it to a list. We then check whether there really is something in the list and return an Option[Bid]. Note that this can be written more concise, but this nicely explains the steps you need to take.

The next function is create

def create(bid: Bid): Bid = {
      var id: Long = -1;
 
      // start a db session
      db withSession {
        // create a new bid
        val res = BidMapping.noID insert (bid.forItem.longValue, bid.minimum.doubleValue, bid.maximum.doubleValue, bid.currency, bid.bidder.toLong, System.currentTimeMillis());
        // get the autogenerated bid
        val idQuery = Query(SimpleFunction.nullary[Long]('LASTVAL'));
        id = idQuery.list().head;
      }
      // create a bid to return
      val createdBid = new Bid(Option(id), bid.forItem, bid.minimum, bid.maximum, bid.currency, bid.bidder, bid.date);
      createdBid;
    }

We now use the custom BidMapping ‘constructor’ noID to generate an insert statement. If we didn’t specify noID we are required to already specify an id. Now that we’ve inserted a new Bid object in the database, we need to return the just created Bid, with the new id, to the user. For this we need to execute a simple query called ‘LASTVAL’, which returns the last autogenerated value. In our case, this is the id of the bid that was created. From this information we create a new Bid, which we return.

The last operation for our repository is the delete function. This function first checks whether the specified bid is present, and if it is, it deletes it.

    def delete(user:String, bid: Long) : Option[Bid] = {
      // get the bid we're deleting
      val result = get(bid,user);
 
      // delete the bid
      val toDelete = BidMapping where (_.id === bid)
      db withSession {
        toDelete.delete
      }
 
      // return deleted bid
      result
    }

Here we use the ‘where’ filter to create the query we want to execute. When we call delete on this filter all matching elements are deleted. And that’s the most basic use of scalaquery for persistency. If you need more complex operations (like joins) look at the scalaquery.org website for examples.

We now have functionality to create and delete bids. So it would also be nice if we have some way to authenticate our users. For this tutorial we’re going to create a very simple API Key based authentication scheme. For every request the user has to add a specific header with its API key. Then we can use the information from this key to determine who this user is, and whether he can delete or access specific information.

Security

We’ll start with the key generation part. When someone wants to use our API we require them to specify an application name and the hostname from which the request will be made. This information we’ll use to generate a key they have to use in each request. This key is just a simple HMAC hash.

package org.smartjava.scalatra.util
 
import javax.crypto.spec.SecretKeySpec
import javax.crypto.Mac
import org.apache.commons.codec.binary.Base64
 
object SecurityUtil {
 
  def calculateHMAC(secret: String, applicationName: String , hostname: String ) : String  = {
    val signingKey = new SecretKeySpec(secret.getBytes(),'HmacSHA1');
    val mac = Mac.getInstance('HmacSHA1');
    mac.init(signingKey);
    val rawHmac = mac.doFinal((applicationName + '|' + hostname).getBytes());
 
    new String(Base64.encodeBase64(rawHmac));
  }
 
  def checkHMAC(secret: String, applicationName: String, hostname: String, hmac: String) : Boolean  = {
    return calculateHMAC(secret, applicationName, hostname) == hmac;
  }
 
  def main(args: Array[String]) {
    val hmac = SecurityUtil.calculateHMAC('The passphrase to calculate the secret with','App 1','localhost');
    println(hmac);
    println(SecurityUtil.checkHMAC('The passphrase to calculate the secret with','App 1','localhost',hmac));
  }
}

The above helper object is used to calculate the initial hash we send to the user and can be used to validate an incoming hash. To use this in our REST API we need to intercept all the incoming requests and check these headers before invoking the specific route. With scalatra we can do this by using the before() function:

package org.smartjava.scalatra.routes
import org.scalatra.ScalatraBase
import org.smartjava.scalatra.repository.KeyRepository
 
/**
 * When this trait is used, the incoming request
 * is checked for authentication based on the
 * X-API-Key header.
 */
trait Authentication extends ScalatraBase {
 
  val ApiHeader = 'X-API-Key';
  val AppHeader = 'X-API-Application';
  val KeyChecker = new KeyRepository; 
 
  /**
   * A simple interceptor that checks for the existence 
   * of the correct headers
   */
  before() {
    // we check the host where the request is made
    val servername = request.serverName;
    val header = Option(request.getHeader(ApiHeader));
    val app = Option(request.getHeader(AppHeader));
 
    List(header,app) match {
      case List(Some(x),Some(y)) => isValidHost(servername,x,y);
      case _ => halt(status=401, headers=Map('WWW-Authenticate' -> 'API-Key'));
    }
  }
 
  /**
   * Check whether the host is valid. This is done by checking the host against
   * a database with keys.
   */
  private def isValidHost(hostName: String, apiKey: String, appName: String): Boolean = {
    KeyChecker.validateKey(apiKey, appName, hostName);
  }
 
}

This trait, which we include in our main scalatra servlet, gets the correct information from the request and checks whether the supplied hash corresponds to the one generated by the code you saw previously. If this is the case the request is passed on, if not, we halt the processing of the request and send back a 401 explaining how to authenticate with this API.

If a client omits these headers he’ll get this as a response: