Using the Testing Harness for Neo4j Extensions

harness

I’ve been creating both unit tests and integration tests for Neo4j Unmanaged Extensions for far too long. The Neo4j Testing Harness was introduced in version 2.1.6 to simplify our lives and just do integration tests. Let’s try it on and see just how awesome we look. First thing we need to do is add the dependency to our project:

 
<dependency>
   <groupId>org.neo4j.test</groupId>
   <artifactId>neo4j-harness</artifactId>
   <version>${neo4j.version}</version>
   <scope>test</scope>
</dependency>

Before we can test anything, we need to write some functionality. We’ll start with a simple method to get the friends of a user. We’ll pass in a user_id, find the node by that key/value pair, traverse to all the friends and get their properties. The code is pretty straight forward.

 
    @GET
    @Path("/friends/{user_id}")
    @Produces({"application/json"})
    public Response Friends(
            @PathParam("user_id") final String userId,
            @Context final GraphDatabaseService db) throws IOException {
        List<Map<String, Object>> results = new ArrayList<>();

        try (Transaction tx = db.beginTx()) {
            final Node user = db.findNode(Labels.User, "user_id", userId);

            if (user != null) {
                for (Relationship r : user.getRelationships(Direction.OUTGOING, RelationshipTypes.FRIENDS)) {
                    Node friend = r.getEndNode();
                    HashMap properties = new HashMap();
                    for (String key : friend.getPropertyKeys()) {
                        properties.put(key, friend.getProperty(key));
                    }
                    results.add(properties);
                }
            }
        }

        return Response.ok().entity(objectMapper.writeValueAsString(results)).build();
    }

Now to test it. We are going to use the JUnit library and create a new @Rule that builds a Neo4j test service initialized with a Cypher Statement along with some test data and the rest end point for our class:

 
    @Rule
    public Neo4jRule neo4j = new Neo4jRule()
            .withFixture(CYPHER_STATEMENT)
            .withExtension("/v1", Service.class);

The Cypher Statement creates a user and 3 friends which we will use for our test:

 
private static final String CYPHER_STATEMENT =
            new StringBuilder()
                    .append("CREATE (user1:User {user_id:'u1', name:'Max'})")
                    .append("CREATE (friend1:User {user_id:'f1', name:'Michael'})")
                    .append("CREATE (friend2:User {user_id:'f2', name:'Peter'})")
                    .append("CREATE (friend3:User {user_id:'f3', name:'David'})")
                    .append("CREATE (user1)-[:FRIENDS]->(friend1)")
                    .append("CREATE (user1)-[:FRIENDS]->(friend2)")
                    .append("CREATE (user1)-[:FRIENDS]->(friend3)")
                    .toString();

Our actual test will use the rule and ask for a response. We’ll convert the array we get from Neo4j into a Set since we don’t care about the order our friends came back in, just that they all come back, and assert that they are equal.

 
    @Test
    public void shouldRespondToFriends() throws IOException {
        HTTP.Response response = HTTP.GET(neo4j.httpURI().resolve("/v1/service/friends/u1").toString());
        ArrayList actual = response.content();

        HashSet expectedSet = new HashSet<>(expected);
        HashSet actualSet = new HashSet<>(actual);

        assertTrue(actualSet.equals(expectedSet));
    }

Now we can run our test and see that everything passes.

Screen Shot 2015-07-05 at 10.18.47 AM

If you take a closer look, you’ll see that it actually started a Neo4j server and called the rest endpoint. Now, we should be running our tests all the time, specially with every commit, but sometimes I forget to do so. Let’s fix my forgetfulness with some automation and have the tests run on Travis-CI.

Octocat travis-mascot-200px logotravis+github

The CI in Travis-CI stands for “Continuous Integration” and it hooks right into GitHub to make our lives easy. Every time we push to GitHub, Travis will build our project and run it’s tests. To make this happen, you need to sign up and enable it on both services and then create a “.travis.yml” file that can look as simple as:

 
language: java
jdk:
  - oraclejdk8

Travis uses smart defaults, so if you are using Maven, Gradle or Ant you’ll be just fine. See the “building a Java project” documentation for more details. We’ll commit this file to our project and push it to our github repo and let the magic happen:

Screen Shot 2015-07-05 at 10.30.46 AM

We are proud of our passing tests, so let’s tell everyone about it. We’ll modify our README.md and add the following:

 
- [![Build Status](https://secure.travis-ci.org/maxdemarzi/neo_travis.png?branch=master)](http://travis-ci.org/maxdemarzi/neo_travis)

This will pull an image from travis with either passing or failing, depending on the status of the last run of our build.

Screen Shot 2015-07-05 at 10.40.32 AM

So now let’s go back to our extension. We are getting a users friends, and the code we wrote is fine if a user has a few friends and they don’t have a ton of properties on their nodes. But what if the opposite is true? Say a user with 5000 friends and lots of properties. Our code would build all that up in memory an array of hashmaps, then convert it to a string and send it off as JSON. Mark Needham wrote about how this is a bad idea and would hurt your performance, but we see it done time and time again (I’m guilty too). So let’s try producing streaming output instead. Here we will use a JsonGenerator, start an array and then for each friend create an object with fields for each property. Finally we will end the array, flush our stream and close the JsonGenerator:

 
    @GET
    @Path("/friends2/{user_id}")
    @Produces({"application/json"})
    public Response Friends2(
            @PathParam("user_id") final String userId,
            @Context final GraphDatabaseService db) throws IOException {

        StreamingOutput stream = new StreamingOutput() {
            @Override
            public void write(OutputStream os) throws IOException, WebApplicationException
            {
                JsonGenerator jg = objectMapper.getJsonFactory().createJsonGenerator( os, JsonEncoding.UTF8 );
                jg.writeStartArray();

                try (Transaction tx = db.beginTx()) {
                    final Node user = db.findNode(Labels.User, "user_id", userId);

                    if (user != null) {
                        for (Relationship r : user.getRelationships(Direction.OUTGOING, RelationshipTypes.FRIENDS)) {
                            Node friend = r.getEndNode();
                            jg.writeStartObject();
                            for (String key : friend.getPropertyKeys()) {
                                jg.writeObjectField(key, friend.getProperty(key));
                            }
                            jg.writeEndObject();
                        }
                    }
                }
                    jg.writeEndArray();
                    jg.flush();
                    jg.close();
                }
            };

        return Response.ok().entity(stream).type(MediaType.APPLICATION_JSON).build();
    }

Our test for this change looks almost identical to test 1, except we are using “friends2”:

 
    @Test
    public void shouldRespondToFriends2() throws IOException {
        HTTP.Response response = HTTP.GET(neo4j.httpURI().resolve("/v1/service/friends2/u1").toString());
        ArrayList actual = response.content();

        HashSet expectedSet = new HashSet<>(expected);
        HashSet actualSet = new HashSet<>(actual);

        assertTrue(actualSet.equals(expectedSet));
    }

We can go ahead and commit this change, push it to github and watch our tests run automatically. As always, you will find the code on github, and you can see the passing tests on travis-ci.

 
Results :
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 20.295 s
[INFO] Finished at: 2015-07-05T05:39:11+00:00
[INFO] Final Memory: 14M/339M
[INFO] ------------------------------------------------------------------------
The command "mvn test -B" exited with 0.
Done. Your build exited with 0.
Tagged , , , , , , ,

3 thoughts on “Using the Testing Harness for Neo4j Extensions

  1. Thanks for your blog post Max, really cool. My first experiences of testing unmanaged extensions (before using the GA framework because I want to understand how it works under the hood) were a bit painful, testing result of a api response was ok, but testing that the database contains changes or created data in tests was not so obvious, a big PR from Stefan added some useful things to harness, just I think it’s worth to mention it here as comment https://github.com/neo4j/neo4j/pull/4330 .

    Cheers,

    Chris

  2. […] Using the Testing Harness for Neo4j Extensions by Max de Marzi […]

  3. maxdemarzi says:

    I’ve edited this repo to also use Coveralls to get test coverage stats.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: