Building well tested applications with SpringBoot / Gradle / Cucumber / Gatling

I am a huge test first advocate. Since seeing Martin Thompson speak I am now trying to include performance testing with the same approach. I am going to call this approach Performance, Acceptance and Unit Test Driven Development, or PAUTDD :)

Tools/frameworks/library come and go so I'll start with the theory then show how I set this up using Junit and Mockio for unit testing, Cucumber for acceptance tests and Gatling for performance tests. Well I won't show JUnit and Mockito because that is boring!

So here's how I develop a feature:

1. Write a high level end to end acceptance test. There will be times where I'll want acceptance tests not to be end to end, like if there was an embedded rules engine.
2. Write a basic performance test.
3. TDD with unit testing framework until the above two pass.
4. Consider scenario based performance test.

I hate to work without test first at the acceptance level, even for a small feature (half a day dev?) I find them invaluable for keeping me on track and letting me know when functionally I am done. Especially if I end up doing a bit too much Unit testing/Mocking (bad Chris!) as when head down in code it is easy to forget the big picture: what functionality are you developing for a user. 

Next is a newer part of my development process. Here I want a performance test for the new feature, this may not be applicable but it usually is. Which ever framework I am using here I want to be able to run it locally and as part of my continuous integration for trend analysis. However I want more for my effort than that, I really want to be able to use the same code for running against various environments running different hardware.

I hate performance tests that aren't in source control and versioned. Tools that use a GUI are no use to me, I constantly found while working at my last company that the "performance testers" would constantly change their scripts and this would change the trend way more than any application changes. I want to be able to track both.

So how to set all this up for a Spring Boot application using Cucumber and Gatling?

This post is on build setup, not the actual writing of the tests. My aim is to enable easy acceptance/performance testing.

Here's the layout of my latest project:

Main is the source, test is the unit tests and e2e is both the Cucumber and the Gatling tests. I could have had separate source sets for the Cucumber and Gatling tests but that would have confused IntelliJ's Gradle support too much (and they are nicely split by Cucumber being Java and Gatling being Scala).

Cucumber JVM

There are various articles on Cucumber-JVM, here's the steps I used to get this running nicely in the IDE and via Gradle.

First the new source set:

Nothing exciting here, we are using the same classapth as test, we could have had a separate one.

Next is dependencies, this is actually the Gatling and HTTP client (for hitting our application) as well.

We cucumber JUnit and Spring dependencies.

Next is the source code for the acceptance tests. The Features are in the resource folder and the source is in the Java folder. To allow running via Gradle you also create a JUnit test to run all the features. Intellij should work find without this by just running the feature files.

Here I have Features separated by type, here's an example Feature:

I like to keep the language at a high level so a non-techy can write these. The JUnit test RunEndToEndTests looks like this:


This is what Gradle will pick up when we run this from the command line. You could separate this out into multiple tests if you wanted.

For running inside IntelliJ you might need to edit the run configuration to include a different Glue as by default it will be the same as the package your Feature file is in, for this project this wouldn't pick up the GlobalSteps as it is outside of the security/users folder. This is what my configuration looks like, I set this as the default:

Now our features will run if you want to see what the implementation of the Steps look like, checkout the whole project from Github.

Gatling

This is my first project using Gatling, I wanted my scenarios in code that I could have in version control. Previously I've used JMeter. Where as you can checkin the XML it really isn't nice to look at in diff tools. I've also been forced *weep* to use more GUI based tools like SOASTA and HP Load runner. One thing I haven't looked at is Gatling's support for running many agents. For me to continue using Gatling beyond developer trend analysis this needs to be well supported.

We already have the dependencies, see the dependencies section above, and we're going to use the same source set. The only difference is we're going to be writing these tests in Scala.

My first requirement was not to have to have Gatling installed manually on developer and CI boxes. Here's how to do this in Gradle:

Where BasicSimulation is the fully qualified name of my Gatling load test. All we do here is define a JavaExec task with the Gatling main class, tell Gatling where our source is and which simulation to run.

To tie all this together so it runs every time we fun Gradle check we add the following at the bottom of our Gradle build file:

This will produce reports that can be published + consumed by the Jenkins Gatling plugin.

To run the same load test from Intellij we do exactly the same in a run configuration:

A basic Gatling tests looks like this:


This test will run a very simple test where a single virtual user hits the /api/auction URL 100 times with a pause of 10 milliseconds. The top couple of lines start the Spring boot application and register a shut down hook to stop it.

We'll then end up with a report that looks like this:

This is a pretty terrible load test as it runs for 2 seconds and has a single user. But the point of this post is to setup everything so when adding new functionality it is trivial to add new performance and acceptance tests.

That's it, happy testing! If you want to see the whole project is on Github. It us under active development so you'll know how I got on with Gatling based if it is still there and there are lots of tests that use it!

Using Gradle as a poor man's Cassandra schema management tool

I work across a desktop and two laptops so reproducible builds mean a lot to me! I often slate Gradle for being buggy and not doing the simple things well (e.g. dependency management for local development).

However it is awesome when you want a quick bit of build logic. I wanted to build my schema for a Cassandra application I am working on to keep my various machines up to date.

So easy in an extensible system like Gradle. I already had my schema creation commands in src/main/resources/schema/tables.cql

I then added a built script dependency to my build.gradle:


Then added a few imports and a couple of nifty tasks:


Of course this relies on one CQL command per line and isn't exactly liquabase but not bad for 10 minutes hacking.

Lots of these hacks can lead to very ugly build scripts so be careful :)

Gradle, Groovy and Java

I've been moving my projects from Maven to Gradle and it has given me my first exposure to Groovy as it's Gradle's language of choice. Gradle is the perfect build tool for Java projects, Groovy projects and polyglot projects containing any combination of JVM languages. To get going with Groovy with Gradle you need to add the Groovy plugin to a Gradle script - this extends the Java plugin:

apply plugin: 'groovy'

Then the Gradle build will look for both Groovy and Java when you build your project:

Christophers-MacBook-Pro:testing chbatey$ gradle build
:compileJava UP-TO-DATE
:compileGroovy UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:jar UP-TO-DATE
:assemble UP-TO-DATE
:compileTestJava UP-TO-DATE
:compileTestGroovy UP-TO-DATE
:processTestResources UP-TO-DATE
:testClasses UP-TO-DATE
:test UP-TO-DATE
:check UP-TO-DATE
:build UP-TO-DATE

BUILD SUCCESSFUL

IntelliJ's support for Gradle has improved vastly as of late but if you experience problems revert to the Gradle idea plugin instead - I tend to use it if the Gralde project is more than a simple Groovy/Java project. Importing a project with just the above in your Gradle script and having created source folders for Java and Groovy IntelliJ should recognise everything:

Now lets create some Groovy and Java! To allow us to interchange between Java and Groovy and be able to add some unit tests, add the following to the build.gradle:

repositories {

    mavenCentral()

}

dependencies {

    compile 'org.codehaus.groovy:groovy:2.1.6'

    testCompile 'junit:junit:4.+'

}

Now lets see a Groovy class testing a Java class:

On the left we can see a Java class in the main source tree and on the left a Groovy class testing it. This is a nice way to learn some Groovy and write tests with less boilerplate. You can say goodbye to types and semicolons. It is an easy transition for a Java developer as all Java is valid Groovy so you can write Java and Groovyify it as much or as little as you like. There is nothing stopping you from adding production Groovy code and testing it with Java as well.

If you want to stay on the command line then just run gradle test from the command line. If you want the source it's on GitHub: https://github.com/chbatey/GroovyTestingJava

From Spring to Guice

For the past two and a half years I have worked primarily on standalone Java applications using Spring/Relational databases and messaging software such as Websphere MQ.

The Well Grounded Java Developer by Ben Evans and Martijn Verburg mentions Guice as an alternative to Spring for dependency injection. I like learning new technologies so I've decided to give it ago!

Here's my attempt at getting a little Guice application up and running. Firstly, I'm going to use Gradle and I'm going to get the Guice jars from the maven central repo. Here is my gradle build file:


apply plugin: 'java'
apply plugin: 'idea'

repositories {
  mavenCentral()
}

dependencies {
  compile 'com.google.inject:guice:3.0'
  testCompile 'junit:junit:4.11'
  testCompile 'org.mockito:mockito-all:1.9+'
}

I've also applied the idea plugin as I am trying out IntelliJ this week after seven years of Eclipse!

So I want a simple application that stores expenses. I am currently spending far too much money on coffee and want to start tracking it.

Lets start with a simple interface:

package com.batey.expense.store;

public interface ExpenseStorer {

    void storeExpense(Expense e);

    Expenses lookupExpenses(String name);

}

With a simple implementation:

package com.batey.expense.store;

import com.google.inject.Singleton;

import org.slf4j.Logger;

import org.slf4j.LoggerFactory;

import java.util.HashMap;

import java.util.Map;

@Singleton

public class InMemoryExpenseStorer implements ExpenseStorer {

    private static final Logger LOGGER = LoggerFactory.getLogger(InMemoryExpenseStorer.class);

    private static final Expenses NULL_EXPENSE = new Expenses();

    private Map<String, Expenses> expenses = new HashMap<>();

    @Override

    public void storeExpense(Expense e) {

        Expenses expenses1 = expenses.get(e.getName());

        if (expenses1 == null) {

            expenses1 = new Expenses();

            expenses.put(e.getName(), expenses1);

            expenses1.setGroupName(e.getName());

        }

        expenses1.getExpenses().add(e);

        LOGGER.info("Got {}", expenses1);

        LOGGER.info("Updated expenses {}", expenses);

    }

    @Override

    public Expenses lookupExpenses(String name) {

        Expenses expenses1 = expenses.get(name);

        LOGGER.info("Current expenses {}", expenses);

        if (expenses1 == null) {

            expenses1 = new Expenses();

            expenses1.setGroupName(name);

            expenses.put(name, expenses1);

        }

        LOGGER.info("Returning {}", expenses1);

        return expenses1;

    }

}

Now I want to use this class in another part of my system. Lets say we have another class called BudgetTracker:

package com.batey.expense.budget;

import com.batey.expense.store.ExpenseStorer;

import com.batey.expense.store.Expense;

import com.batey.expense.store.Expenses;

import com.google.inject.Inject;

import java.math.BigDecimal;

public class BudgetTracker {

    @Inject

    private ExpenseStorer expenseStorer;

    public void buySomething(String description, String who, BigDecimal howMuch) {

        Expense expense = new Expense();

        expense.setAmount(howMuch);

        expense.setName(who);

        expense.setDescription(description);

        expenseStorer.storeExpense(expense);

    }

    public BigDecimal howMuchHaveISpent(String who) {

        Expenses expenses = expenseStorer.lookupExpenses(who);

        return expenses.getTotal();

    }

}

Everything above was done without thinking about dependency injection. Then rather than instantiate an ExpenseStorer directly I then read through Google's getting started guide at http://code.google.com/p/google-guice/wiki/GettingStarted and added the Guide annotations in red.

So how do we run this? For that we need a Guide module:

package com.batey.expense.store;

import com.google.inject.AbstractModule;

public class ExpenseModule extends AbstractModule {

    @Override

    protected void configure() {

        bind(ExpenseStorer.class).to(InMemoryExpenseStorer.class);

    }

}

And then I put together a very simple main method:

package com.batey.expense.budget;

import com.batey.expense.store.ExpenseModule;

import com.google.inject.Guice;

import com.google.inject.Injector;

import java.math.BigDecimal;

public class Main {

    public static final void main(String[] args) {

        Injector injector = Guice.createInjector(new ExpenseModule());

        BudgetTracker budgetTracker = injector.getInstance(BudgetTracker.class);

        budgetTracker.buySomething("A tv", "Chris", new BigDecimal("100.00"));

        budgetTracker.buySomething("A cake", "Chris", new BigDecimal("50.00"));

        System.out.println("Chris has spent: " + budgetTracker.howMuchHaveISpent("Chris"));

    }

}

And then running this:

20:39:46.168 [main] INFO  c.b.e.store.InMemoryExpenseStorer - Got Expenses{expenses=[Expense{name='Chris', amount=100.00, description='A tv'}], total=0, groupName='Chris'}

20:39:46.171 [main] INFO  c.b.e.store.InMemoryExpenseStorer - Updated expenses {Chris=Expenses{expenses=[Expense{name='Chris', amount=100.00, description='A tv'}], total=0, groupName='Chris'}}

20:39:46.172 [main] INFO  c.b.e.store.InMemoryExpenseStorer - Got Expenses{expenses=[Expense{name='Chris', amount=100.00, description='A tv'}, Expense{name='Chris', amount=50.00, description='A cake'}], total=0, groupName='Chris'}

20:39:46.172 [main] INFO  c.b.e.store.InMemoryExpenseStorer - Updated expenses {Chris=Expenses{expenses=[Expense{name='Chris', amount=100.00, description='A tv'}, Expense{name='Chris', amount=50.00, description='A cake'}], total=0, groupName='Chris'}}

20:39:46.172 [main] INFO  c.b.e.store.InMemoryExpenseStorer - Current expenses {Chris=Expenses{expenses=[Expense{name='Chris', amount=100.00, description='A tv'}, Expense{name='Chris', amount=50.00, description='A cake'}], total=0, groupName='Chris'}}

20:39:46.172 [main] INFO  c.b.e.store.InMemoryExpenseStorer - Returning Expenses{expenses=[Expense{name='Chris', amount=100.00, description='A tv'}, Expense{name='Chris', amount=50.00, description='A cake'}], total=0, groupName='Chris'}

Chris has spent: 150.00

Key points:

• We use the @Singleton on the ExpenseStorer to say we only want one of these in our application. Singleton is the default in spring: not so in Guice.
• We use @Inject to identify a field needs injected. This could have also been done via the constructor. This is like the spring @Autowired
• Inside the ExpenseModule we bind any injected ExpenseStorers to the InMemoryExpenseStorer implementation.

Success! From never having used Guice getting this together was very straightforward and took under thirty minutes. My initial thoughts of Guice for dependency injection:

• It is very simple! I don't remember how long my first Spring application took me to get going but I bet it was longer than this!
• No XML! Some people love configuration XML; others hate it. Personally, I'm sitting on the fence. In the latest applications I've written I've used a combination of annotations Java config along with XML config. Guice is pure Java config and I'd have to see a large scale project using it before passing judgement on that.
• Standardisation! Guice is the first implementation of JSR330 - making DI standardized in Java can only be a good thing.

I'm going to keep building my expense application with Guice and will post more about it when I have used it more!