Using Stubbed Cassandra's new JUnit rule

In my first post about unit testing Java applications that use Cassandra you had to start Scassandra before your test, reset it between tests and finally stop it before it after your tests. Since then I've accepted a pull request which hides all this away in a JUnit rule.

So the old code in your tests looked like this for starting Scassandra:


This for stopping Scassandra:


And this for resetting Scassandra between tests:


Now you can simply do this:


Assigning it to a ClassRule means that it will start Scassandra before all your tests and stop it after all your tests. Then assigning it to a regular Rule means that the Activity client and Priming client will be reset between tests.

Unit testing Java Cassandra applications

I often get asked what is the best method for unit and and integration testing Java code that communicates with Cassandra.

Here are what I think the options are:

1. Mocking libraries: Use mocking libraries such as Mockito to mock out the driver you are using
2. Cassandra Unit
3. Integration test the class in question with against a real Cassandra instance running locally
4. Stubbed Cassandra (disclaimer: I made this)

Lets take these one by one and look at their respective advantages and disadvantages. The things to consider are:

• Speed of the tests
• Able to run the tests concurrently and how easy this is to achieve
• Able to test everything? Even failures?
• Are the tests brittle?
• Readability of the tests
• Requirements on environment e.g do you need a Cassandra instance installed?
• Confidence it will work against a real Cassandra
• Subjective nonsense by writer of this blog

1 Mock the library

You can use a combination of factories and mocking to stop the driver actually interacting with Cassandra. Then verify your code's interactions with these mocks.

Advantages:

• Fast - no I/O
• Execute tests concurrently
• No Cassandra instance required on machine your code compile runs
• Can test everything including the various failures as you can mock the library to throw ReadTimeout exceptions etc

Disadvantages:

• You may mock the driver to behave in a different way to how it will behave 
• Very hard to understand tests due to large amount of boiler plate mocking code
• Very brittle tests. Change your driver, change your tests! Change from a query to a prepared statement, change your tests!
• A lot of boiler plate. Take the Datstax driver for example, it returns a ResultSet, which is iterable, fancy writing the code that mocks it returning many results?

Conclusion:

• Don't do it if you wish to remain sane

2 Cassandra Unit

Cassandra Unit is a tool for starting an embedded Cassandra in the JVM your tests are running. It also has a great API for ingesting data into Cassandra for your tests.

Advantages:

• Pretty fast. It is all in process.
• Can run tests concurrently if they use different keyspaces and none of the tests turn it off etc
• Can use CQL to load data as it is a real Cassandra. I think this leads to readable tests
• No Cassandra required on machine. Can use the it via a Maven dependency
• High confidence your code will work against a real Cassandra 

Disadvantages:

• Unable to test failure scenarios. What is a read time out when there is only one node running in the same JVM as your test?
• No way to verify consistency of queries

Conclusion:

• Very useful tool for in process happy path tests

3 Integration style tests using a real Cassandra

This is something I've done a lot of recently. Every dev machine at my current work place has a Cassandra instance running (using the awesome tool ccm). Then we test our DAOs by assuming it is there and doing testing in a dynamic keyspace.

Advantages:

• Can run tests concurrently if they use different keyspaces and none of the tests turn it off etc
• Tests aren't brittle, can change from queries to prepared statements, or the queries involved without changing tests
• Readable tests - all data setup is done in CQL
• Very high confidence it will work against a real Cassandra as the test is against a  real Cassandra :)

Disadvantages:

• Probably the slowest option. But it is still millisecond quick.
• Hard to test scenarios other than turning the node off. This then makes the tests slow.
• Cassandra required to build and run tests

Conclusion:

• Slightly slower but very useful for testing happy paths
• Very similar to using Cassandra unit

4 Stubbed Cassandra

Stubbed Cassandra is a new open source tool that pretends to be Cassandra and can be primed to returns rows, read timeouts, write timeouts and unavailable errors. It can be used via a maven dependency or as a standalone server.

Advantages:

• Very fast
• Can test all types of errors and be confident in what the driver does as the driver thinks it is a real Cassandra
• Can run many instances inside the same JVM listening on different binary ports. So can run tests concurrently with no extra effort e.g no requirement to use different keyspaces
• Tests less brittle than mocking the driver. Can change driver without changing test but if you change queries you need to update your priming
• No requirement to have a real Cassandra. Just brought in by a maven dependency

Disadvantages:

• Slightly more brittle than a real Cassandra/Cassandra unit. Requires priming on the query, priming for each prepared statement
• Slightly less confidence it will work against a real Cassandra as it isn't a real Cassandra. But more confidence than mocking
• It is new and does not support all of Cassandra's features, so if you use a feature that Scassandra doesn't support you are stuck!

Conclusion:

• Best solution for all error case testing
• Best solution if you need to execute tests concurrently

Using Stubbed Cassandra: Unit testing Java applications

My first article on Scassandra introduced what it is and why I've made it.

This article describes how to use Scassanda to help unit test a Java class that stores and retrieves data from Cassandra.

It assumes you're using a tool that can download dependencies from maven central e.g Maven, Gradle or SBT.

First add Scassandra as a dependency. It is in maven central so you can add it to your pom with the following xml:

<dependency>
  <groupId>org.scassandra</groupId>
  <artifactId>java-client</artifactId>
  <version>0.2.1</version>
</dependency>

Or the following entry in your build.gradle:

dependencies {
    compile('org.scassandra:java-client:0.2.1')
}

There are four important classes you'll deal with from Java:

• ScassandraFactory - used to create instances of Scassandra
• Scassandra - interface for starting/stopping Scassandra and getting hold of a PrimingClient and an ActivityClient
• PrimingClient - sends priming requests to Scassandra RESTful admin interface
• ActivityClient - retrieves all the recorded queries and prepared statements from the Scassandra RESTful admin interface

The PrimingClient and ActivityClient have been created to ease integration for Java developers. Otherwise you would need to construct JSON and send it over HTTP to Scassandra.

You can start a Scassandra instance per unit test and clear all primes and recorded activity between tests.

To start Scassandra before your test starts add a BeforeClass e.g:


You can also add a AfterClass to close Scassandra down:

Now that you have Scassandra running lets write a test. Perhaps you want to test a simple Java DAO that connects to Cassandra and executes a query.

And you have a backing table like:

CREATE TABLE person (
  id int,
  first_name text,
  PRIMARY KEY (id)
)

Lets TDD the DAO using Scassandra starting with our connect method:

Lets look at what this code is doing:

• Line 4: Informs the activity client to clear all recorded connections. This is to stop other tests that have caused connections interfering with this one. 
• Line 6: We call on connect on our PersonDao.
• Line 8: We call retrieveConnections on the activity client and expect there to be at least one. The Java Datastax driver makes multiple connections on startup so you can't assert for this to be 1.

This fails with the following message:

java.lang.AssertionError: Expected at least one connection to Cassandra on connect

at org.junit.Assert.fail(Assert.java:88)

at org.junit.Assert.assertTrue(Assert.java:41)

at com.batey.examples.scassandra.PersonDaoTest.shouldConnectToCassandraWhenConnectCalled(PersonDaoTest.java:94)

Now lets write some code to make it pass:

Now lets test the retrieveNames function gets all the first_names out of the person table.

This will prime Scassandra to return a single row with the column first_name with the value Chris. We expect our DAO to turn that into a List of strings containing Chris. To make this pass we need to execute a query and convert the ResultSet, something like this:

Next lets say you have the requirement that you really must not get an out of date list of names. So you want to test that the consistency you do the query is QUORUM. You can test this like this:

Lets look at what each line is doing:

• Line 4 builds the expected query, note the consistency is also set. If you build a Query without a consistency it defaults to ONE.
• Line 7 clears all the recorded activity so that another test does not interfere with this one. It also clears the queries that were executed as part of connect (the Datastax Java driver issues quite a few queries on the system keyspace on startup)
• Line 11 retrieves all the queries your application has execited
• Line 12 verifies the expected query that was built on Line 4 has been executed

This will fail with an error message like this:

java.lang.AssertionError: Expected query with consistency QUORUM, found following queries: [ {Query{query='select * from people', consistency='ONE'}]

We can make this pass by adding the consistency to our query:

And we're done!

This has been a brief instruction to Scassandra but hopefully the above gives you an idea of how Scassandra can be used to test your Java applications that use Cassandra. We've covered:

• Priming basic queries
• Verifying queries
• Verifying connections

Future blog posts will show you how to:

• Prime prepared statements
• Prime different column types in responses
• Prime error cases

Scassandra has only just been released. The future road map includes:

• JUnit rule so you don't need to handle starting/stopping and clearing recorded activity
• More generic priming e.g any query on this table
• Support for more drivers

All the code for this example can be found in full here.