This article is part of our Academy Course titled Developing Modern Applications with Scala.
In this course, we provide a framework and toolset so that you can develop modern Scala applications. We cover a wide range of topics, from SBT build and reactive applications, to testing and database acceess. With our straightforward tutorials, you will be able to get your own projects up and running in minimum time. Check it out here!
1. Introduction
In this section of the tutorial we are going to talk about testing frameworks which are widely adopted by majority of the Scala application developers. Although the hot debates around effectiveness and usefulness of the test-driven development (or just TDD) practices are going on for years, this section is based on a true belief that tests are great thing which makes us better software developers and improves the quality and maintainability of the software systems we have worked or are working on.
Why to talk about testing so early? Well, most of the future sections will introduce us to different frameworks and libraries, and as a general rule we are going to spend some time discussing the testing strategies applicable in their contexts. Plus, it is much easier and safer way to show off a new language or framework capabilities to your team (or organization) when it does not affect production code anyhow.
Table Of Contents
2. ScalaCheck: the Power of Property-Based Testing
The first framework we are going to touch upon is ScalaCheck which serves the purpose of automated property-based testing. In case you are not familiar with property-based testing, it is quite intuitive and powerful technique: it basically verifies (or better to say, checks) that assertions about the output results of your code hold true for a number of auto-generated inputs.
Let us start from very simple example of Customer case class so the idea and benefits of property-based testing becomes apparent.
case class Customer(firstName: String, lastName: String) {
def fullName = firstName + " " + lastName
}Basically, the traditional way of ensuring that full name is assembled from first and last names would be to write a test case (or multiple test cases) by explicitly providing first and last name values. With ScalaCheck (and property-based testing) it is looking much simpler:
object CustomerSpecification extends Properties("Customer") {
property("fullName") = forAll { (first: String, last: String) =>
val customer = Customer(first, last)
customer.fullName.startsWith(first) &&
customer.fullName.endsWith(last)
}
}The inputs for first and last names are automatically generated, while the only thing we need to do is to construct the Customer class instance and define our checks which must be true for all possible inputs:
customer.fullName.startsWith(first) && customer.fullName.endsWith(last)
Looks really simple, but we can do even better than that. ScalaCheck by default provides the generators for all primitive types, giving the option to define your own. Let us go one step further and define the generator for Customer class instances.
implicit val customerGen: Arbitrary[Customer] = Arbitrary {
for {
first <- Gen.alphaStr
last <- Gen.alphaStr
} yield Customer(first, last)
}
With this generator in place, our test case becomes even more trivial and readable:
property("fullName") = forAll { customer: Customer =>
customer.fullName.startsWith(customer.firstName) &&
customer.fullName.endsWith(customer.lastName)
}
Very elegant, isn’t it? Surely, ScalaCheck has much more to offer in terms of supporting flexible checks (and verification failure reporting) for quite complex test scenarios as we are going to see in the next sections of this tutorial. Nonetheless, the official documentation has a number of good references in case you would like to learn more about ScalaCheck.
Having test cases is good, but how to run them? There are multiple ways to run ScalaCheck tests but the most convenient one is to use SBT, the tool we have learned about in the first section of this tutorial, using its test task.
$ sbt test ... [info] + Customer.fullName: OK, passed 100 tests. [info] + CustomerGen.fullName: OK, passed 100 tests. [info] Passed: Total 2, Failed 0, Errors 0, Passed 2 [success] Total time: 1 s
As the SBT output shows, ScalaCheck generated 100 different tests out of our single test case definition, saving quite a lot of our time.
3. ScalaTest: Tests as Specifications
ScalaTest is a great example of Scala-based variant of the full-fledged testing framework which could be found in many other languages (like Spock Framework or RSpec just to name a few). In the core of the ScalaTest are test specifications which support different styles of writing tests. In fact, this variance of testing styles is extremely useful feature as it lets the developers coming to Scala universe from other languages to follow the style they might be already familiar and comfortable with.
Before rolling up the sleeves and exploring ScalaTest, it is worth to mention that the current stable release branch is 2.2 but the next version 3.0 is about to be released (hopefully) very soon, being in RC3 stage right now. As such, the latest 3.0-RC3 is going to be the ScalaTest version we will build our test cases upon.
Let us get back to the Customer case class and show off different flavors of ScalaTest specifications, starting from the basic one, FlatSpec:
class CustomerFlatSpec extends FlatSpec {
"A Customer" should "have fullName set" in {
assert(Customer("John", "Smith").fullName == "John Smith")
}
}Although the usage of assertions is known and understandable by most of us, there are better, more human-friendly ways to express your expectations. In ScalaTest there are matchers which serve this purpose. Let us take a look on another version of the test, still using the FlatSpec:
class CustomerMatchersFlatSpec extends FlatSpec with Matchers {
"A Customer" should "have fullName set" in {
Customer("John", "Smith").fullName should be ("John Smith")
}
}With just a minor change where the explicit assert has been replaced by convenient should be matcher, provided by Matchers trait, the test case could be now read fluently, as a story.
Aside from FlatSpec, ScalaTest includes many other favors in a form of FunSuite, FunSpec, WordSpec, FreeSpec, Spec, PropSpec and FeatureSpec. You are certainly encouraged to look on those and pick your favorite but the one we are going to discuss at last is FeatureSpec.
Behavior-driven development (or simply BDD) is another testing methodology which emerged from TDD and became quite popular in the recent years. In BDD the test scenarios (or better to say acceptance criteria) should be written for every feature being developed and should follow Given / When / Then structure. ScalaTest supports this kind of testing with FeatureSpec style. So let us take a look on BDD version of the test scenario we have implemented so far.
class CustomerFeatureSpec extends FeatureSpec with GivenWhenThen with Matchers {
info("As a Customer")
info("I should have my Full Name composed from first and last names")
feature("Customer Full Name") {
scenario("Customer has correct Full Name representation") {
Given("A Customer with first and last name")
val customer = Customer("John", "Smith")
When("full name is queried")
val fullName = customer.fullName
Then("first and last names should be returned")
fullName should be ("John Smith")
}
}
}Certainly, for such a simple test scenario the BDD style may look like overkill, but it is actually very expressive way to write acceptance criteria which could be understood by business and engineering people at the same time.
With no doubts, ScalaTest has first-class SBT support and with all our test suites in place, let us run them to make sure they all pass:
$ sbt test ... [info] CustomerFeatureSpec: [info] As a Customer [info] I should have my Full Name composed from first and last names [info] Feature: Customer Full Name [info] Scenario: Customer has correct Full Name representation [info] Given A Customer with first and last name [info] When full name is queried [info] Then first and last names should be returned [info] CustomerMatchersFlatSpec: [info] A Customer [info] - should have fullName set [info] CustomerFlatSpec: [info] A Customer [info] - should have fullName set [info] Run completed in 483 milliseconds. [info] Total number of tests run: 3 [info] Suites: completed 3, aborted 0 [info] Tests: succeeded 3, failed 0, canceled 0, ignored 0, pending 0 [info] All tests passed. [success] Total time: 1 s
As we can see in SBT output, the FeatureSpec test suite formats and prints out feature and scenario descriptions, plus the details about every Given / When / Then step performed.
4. Specs2: One to Rule Them All
The last (but not least) Scala testing framework we are going to discuss in this section is specs2. From the features perspective, it is quite close to ScalaTest but specs2 takes a slightly different approach. Essentially, it is also based on test specifications, but there are only two of them: mutable (or unit specification) and immutable (or acceptance specification).
Let us see what those two styles really mean in practice by developing same kind of test cases for our Customer case class.
class CustomerSpec extends Specification {
"Customer" >> {
"full name should be composed from first and last names" >> {
Customer("John", "Smith").fullName must_== "John Smith"
}
}
}It looks like quite simple test suite with familiar structure. Not much different from ScalaTest except it extends org.specs2.mutable.Specification class. It is worth to mention that specs2 has extremely rich set of matchers, providing usually different variations of them depending on your preferred style. In our case, we are using must_== matcher.
Another way to create test suites with specs2 is to extend
org.specs2.Specification class and write down the specification as a pure text. For example:class CustomerFeatureSpec extends Specification { def is = s2"""
Customer
should have full name composed from first and last names $fullName
"""
val customer = Customer("John", "Smith")
def fullName = customer.fullName must_== "John Smith"
}This kind of specifications became possible since introduction of the string interpolation capabilities to Scala language and compiler. As you could guess at this point, such relaxed way of defining test specifications is a great opportunity for writing test cases following BDD style. Surely, specs2 enables that but goes even a bit further by providing additional support in term of steps and scenarios. Let us take a look at the example of such test specification.
class CustomerGivenWhenThenSpec extends Specification
with GWT with StandardDelimitedStepParsers { def is = s2"""
As a Customer
I should have my Full Name composed from first and last names ${scenario.start}
Given a customer last name {Smith}
And first name {John}
When I query for full name
Then I should get: {John Smith} ${scenario.end}
"""
val scenario =
Scenario("Customer")
.given(aString)
.given(aString)
.when() { case s :: first :: last :: _ =>
Customer(first, last)
}
.andThen(aString) { case expected :: customer :: _ =>
customer.fullName must be_== (expected)
}
}Essentially, in this case the test specification consists of two parts: the actual scenario definition and its interpretation (sometimes called code-behind approach). The first and last names are extracted from the definition (using step parsers), Customer case class instance is created in background, and at the last step the expected full name is also extracted from the definition and compared with the customer’s one. If you find it a bit cumbersome, this is not the only way to define Given / When / Then style specifications using specs2. But be advised than other alternatives would require you to track and maintain the state between different Given / When / Then steps.
And finishing up with specs2 it would be incomplete not to mention it seamless integration with ScalaCheck framework just by extending org.specs2.ScalaCheck trait, for example:
class CustomerPropertiesSpec extends Specification with ScalaCheck { def is = s2"""
Customer
should have full name composed from first and last names ${fullName}
"""
val fullName: Prop = forAll { (first: String, last: String) =>
val customer = Customer(first, last)
customer.fullName.startsWith(first) &&
customer.fullName.endsWith(last)
}
}
The result is exactly what you would expect from running a pure ScalaCheck: the checks are going to be run against generated first and last names. Using our friend SBT, let us run all the test suites to ensure we did a good job.
$ sbt test ... [info] CustomerPropertiesSpec [info] + Customer should have full name composed from first and last names [info] [info] Total for specification CustomerPropertiesSpec [info] Finished in 196 ms [info] 1 example, 100 expectations, 0 failure, 0 error [info] [info] CustomerSpec [info] [info] Customer [info] + full name should be composed from first and last names [info] [info] [info] Total for specification CustomerSpec [info] Finished in 85 ms [info] 1 example, 0 failure, 0 error [info] [info] CustomerFeatureSpec [info] Customer [info] + should have full name composed from first and last names [info] [info] Total for specification CustomerFeatureSpec [info] Finished in 85 ms [info] 1 example, 0 failure, 0 error [info] [info] CustomerGivenWhenThenSpec [info] As a Customer [info] I should have my Full Name composed from first and last names [info] [info] Given a customer last name Smith [info] And first name John [info] When I query for full name [info] + Then I should get: John Smith [info] [info] Total for specification CustomerGivenWhenThenSpec [info] Finished in 28 ms [info] 1 example, 4 expectations, 0 failure, 0 error [info] Passed: Total 4, Failed 0, Errors 0, Passed 4 [success] Total time: 10 s
The output from SBT console is very similar to what we have seen for ScalaTest, with all test suites details printed out in compact and readable format.
5. Conclusions
Scala is extremely powerful language designated to solve a large variety of the software engineering problems. But as every other program written in a myriad of programming languages, Scala applications are not immutable to bugs. TDD (and recently BDD) practices are widely adopted by Scala developers community, which led to invention of such a great testing frameworks as ScalaCheck, ScalaTest and specs2.
6. What’s next
In the next section of the tutorial we are going to talk about reactive applications, in particular focusing our attention on reactive streams as their solid foundation, briefly touching upon The Reactive Manifesto as well.
