Using Stubbed Cassandra: Unit testing Java applications

My first article on Scassandra introduced what it is and why I've made it.

This article describes how to use Scassanda to help unit test a Java class that stores and retrieves data from Cassandra.

It assumes you're using a tool that can download dependencies from maven central e.g Maven, Gradle or SBT.

First add Scassandra as a dependency. It is in maven central so you can add it to your pom with the following xml:

<dependency>
  <groupId>org.scassandra</groupId>
  <artifactId>java-client</artifactId>
  <version>0.2.1</version>
</dependency>

Or the following entry in your build.gradle:

dependencies {
    compile('org.scassandra:java-client:0.2.1')
}

There are four important classes you'll deal with from Java:

• ScassandraFactory - used to create instances of Scassandra
• Scassandra - interface for starting/stopping Scassandra and getting hold of a PrimingClient and an ActivityClient
• PrimingClient - sends priming requests to Scassandra RESTful admin interface
• ActivityClient - retrieves all the recorded queries and prepared statements from the Scassandra RESTful admin interface

The PrimingClient and ActivityClient have been created to ease integration for Java developers. Otherwise you would need to construct JSON and send it over HTTP to Scassandra.

You can start a Scassandra instance per unit test and clear all primes and recorded activity between tests.

To start Scassandra before your test starts add a BeforeClass e.g:


You can also add a AfterClass to close Scassandra down:

Now that you have Scassandra running lets write a test. Perhaps you want to test a simple Java DAO that connects to Cassandra and executes a query.

And you have a backing table like:

CREATE TABLE person (
  id int,
  first_name text,
  PRIMARY KEY (id)
)

Lets TDD the DAO using Scassandra starting with our connect method:

Lets look at what this code is doing:

• Line 4: Informs the activity client to clear all recorded connections. This is to stop other tests that have caused connections interfering with this one. 
• Line 6: We call on connect on our PersonDao.
• Line 8: We call retrieveConnections on the activity client and expect there to be at least one. The Java Datastax driver makes multiple connections on startup so you can't assert for this to be 1.

This fails with the following message:

java.lang.AssertionError: Expected at least one connection to Cassandra on connect

at org.junit.Assert.fail(Assert.java:88)

at org.junit.Assert.assertTrue(Assert.java:41)

at com.batey.examples.scassandra.PersonDaoTest.shouldConnectToCassandraWhenConnectCalled(PersonDaoTest.java:94)

Now lets write some code to make it pass:

Now lets test the retrieveNames function gets all the first_names out of the person table.

This will prime Scassandra to return a single row with the column first_name with the value Chris. We expect our DAO to turn that into a List of strings containing Chris. To make this pass we need to execute a query and convert the ResultSet, something like this:

Next lets say you have the requirement that you really must not get an out of date list of names. So you want to test that the consistency you do the query is QUORUM. You can test this like this:

Lets look at what each line is doing:

• Line 4 builds the expected query, note the consistency is also set. If you build a Query without a consistency it defaults to ONE.
• Line 7 clears all the recorded activity so that another test does not interfere with this one. It also clears the queries that were executed as part of connect (the Datastax Java driver issues quite a few queries on the system keyspace on startup)
• Line 11 retrieves all the queries your application has execited
• Line 12 verifies the expected query that was built on Line 4 has been executed

This will fail with an error message like this:

java.lang.AssertionError: Expected query with consistency QUORUM, found following queries: [ {Query{query='select * from people', consistency='ONE'}]

We can make this pass by adding the consistency to our query:

And we're done!

This has been a brief instruction to Scassandra but hopefully the above gives you an idea of how Scassandra can be used to test your Java applications that use Cassandra. We've covered:

• Priming basic queries
• Verifying queries
• Verifying connections

Future blog posts will show you how to:

• Prime prepared statements
• Prime different column types in responses
• Prime error cases

Scassandra has only just been released. The future road map includes:

• JUnit rule so you don't need to handle starting/stopping and clearing recorded activity
• More generic priming e.g any query on this table
• Support for more drivers

All the code for this example can be found in full here.

Testing your RESTful services with Groovy, Spock and HTTPBuilder

I like to test the HTTP portion of the RESTful web services I build in isolation. This is because, in the languages I use (Java, Scala), most of the code responsible for taking in HTTP requests and parsing them is very framework specific and simply delegates the actual logic to another part of my application. Ideally I can launch this part of my application in the same way as it is launched in production but with the rest of the application stubbed out.

Recently I've been using Groovy/Spock for these integration tests, here's how for a really simple web service. In this example it's been built with Spring Boot.

Let's not go into the details of how Spring boot work but it is essentially the same application as in Spring's guide.

The web application has a single path: /exampleendpoint which takes in a single query param: input and returns a JSON object with a single field payload with the value: Something really important: with the input appended.

I want to test that the web service takes in HTTP, takes the query parameter out and builds the JSON correctly. With Groovy/Spock/HTTPBuilder here's how it looks:

Let's look at what is going on here:

• In the setup phase we create a RESTClient
• In the exercising phase we make a call out to the web application.
• In the verification phase we check that the response code is 200, the content type is "application/json" and the body is JSON with a single payload field that contains "Something really important: Get a hair cut"

Benefits over writing this in Groovy/Spock over say just using Java/JUnit:

• The HTTP libraries are easier to use with less code
• The assertions are more expressive
• More magic, for instance the second with() block works on a Groovy map that has automagically created

The above example doesn't deal with how to start and stop the web service under test, I left that out as it is very web framework specific. 

What do you need in your build to get this setup? I used Gradle for this example so it is probably easier to just check out the build.gradle, a lot of it can be ignored as it is related to Spring Boot rather than Spock. Here are the important bits:

The plugins:

And the dependencies:

The entire project is here. Happy testing.

Akka: Testing messages sent to an actor's parent

I've blogged previously about testing messages sent back to the sender and to a child actor. The final common scenario I come across is testing actors that send a messages back to its supervisor/parent. If it is in response to a message from the parent you can use the same technique as described in testing messages sent back to the sender.

However, if that is not the case then TestKit has a very simple way you can do it with the TestActorRef and a TestProbe. Any where you have a piece of code that looks like this:

context.parent ! "Any message"

For example:

Then you can test it by passing a TestProbe to your TestActorRef and using the expect* functions on the TestProbe. So to test the actor above your test will look something like this:

As you can see the TestActorRef can take an ActorRef to use as a parent when you use the apply method that takes in a Props.

Full code here. Happy testing.

Testing Scala Futures with ScalaTest 2.0

Recently I was writing integration tests for a library that returned futures. I wanted simple synchronous tests.

Fortunately in the latest release of ScalaTest they have added something to do just that: the ScalaFutures trait

Say you have a class that returns a Future, e.g:

And you want to write a test for this and you want to wait for the future to complete. In regular code you might use a call back or a for comprehension e.g:


This won't work in a test as the test will finish before the future completes and the assertions are on a different thread.

ScalaFutures to the rescue! Upgrade to ScalaTest 2.0+ and mix in the ScalaFutures trait. Now you can test using whenReady:

Or with futureValue:

Happy testing! Full source code here.

Akka: Testing messages sent to child actors

I previously blogged about testing messages sent back to the sender. Another common scenario is testing that an actor sends the correct message to a child. A piece of code that does this might look like:

But how do you test drive this kind of code?

I've adopted the approach of removing the responsibility of creating the child from the ParentActor. To do this you can pass in a factory to the ParentActor's constructor that can be replaced for testing. To do this the ParentActor is changed to look like this:

The ParentActor now takes in a factory function which takes in an ActorFactoryRef and returns a ActorRef. This function is responsible for creating the ChildActor.

childFactory: (ActorRefFactory) => ActorRef

This is then used by the ParentActor as follows:

val childActor = childFactory(context)

You can now inject a TestProbe for testing rather than using a real ActorFactorRef. For example:

And for the real code you can pass in an anonymous function that uses the ActorRefFactory to actually create a real ChildActor:

Here you can see the ActorFactoryRef (context in Actors implement this interface) is used to create an Actor the same way as the original ParentActor did.

Full source code here.

Akka: Testing that an actor sends a message back to the sender

A common pattern when using actors is for actor A to send a message to Actor B and then for Actor B to send a message back.

This can be tested using the Akka testkit along with Scala Test.

Take the following very simple example. We have an Actor called Server which should accept Startup messages and respond with a Ready message when it is ready for clients to send additional messages.

We can start with a noddy implementation that does nothing:

Then write a test to specify the behaviour. Here I've used TestKit along with Scala Test FunSuite.

This test will fail with the following error message:

assertion failed: timeout (3 seconds) during expectMsg while waiting for Ready

As you can probably guess TestKit waited for 3 seconds for a Ready message to be sent back.

To fix the test we add the the following to the Server Actor implementation:

And now the test will pass! The two important things to take note of are that our test case extended from TestKit, this gives you an ActorSystem. And that the test case mixed in the ImplicitSender trait, this allows us to receive messages use the methods like "expectMsg" to assert that the correct message has been received.

Installing Cassandra 2.0 on Ubuntu

Update your apt source list with the following:

sudo vim /etc/apt/sources.list

#Add at the bottom
deb http://www.apache.org/dist/cassandra/debian 20x main
deb-src http://www.apache.org/dist/cassandra/debian 20x main

Run an apt-get update. 

sudo apt-get update

This will give you a warning about not being able to verify the signatures of the apache repos:

GPG error: http://www.apache.org unstable Release:
The following signatures couldn't be verified because the public key is not available:
NO_PUBKEY 4BD736A82B5C1B00

Now do the following for that key:

gpg --keyserver pgp.mit.edu --recv-keys 4BD736A82B5C1B00
gpg --export --armor 4BD736A82B5C1B00 | sudo apt-key add -

Also add this one:

gpg --keyserver pgp.mit.edu --recv-keys 2B5C1B00
gpg --export --armor 2B5C1B00 | sudo apt-key add -

Now run apt-get update again.
sudo apt-get update

The error should be gone. Now check that all is working and UBuntu can see Cassandra 2.0:

apt-cache showpkg cassandra
Package: cassandra
Versions:
2.0.1

Great! Now install it:

sudo apt-get install cassandra

Now start it:
sudo service cassandra start
xss = -ea -javaagent:/usr/share/cassandra/lib/jamm-0.2.5.jar -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms1001M -Xmx1001M -Xmn100M -XX:+HeapDumpOnOutOfMemoryError -Xss256k

Now you can check you can connect:
cqlsh
Connected to Test Cluster at localhost:9160.
[cqlsh 4.0.1 | Cassandra 2.0.1 | CQL spec 3.1.1 | Thrift protocol 19.37.0]
Use HELP for help.

Where is everything?

• Logs: /var/log/cassandra
• Config: /etc/cassandra/
• Data: /var/lib/cassandra

Done!