Max De Marzi

You are probably sick of me saying it, but one of the things I love about Neo4j is that you can customize it any way you want. Extensions, stored procedures, plugins, custom indexes, custom apis, etc. If you want to do it, then you can do it with Neo4j.

So the other day I was like what about this gRPC thing? Many companies standardize their backend using RESTful APIs, others are trying out GraphQL, and some are using gRPC. Neo4j doesn’t support gRPC out of the box, partially because we have our own custom binary protocol “Bolt”, but we can add a rudimentary version of gRPC support quite easily.

Lets build a Neo4j Kernel Extension to handle sending and receiving cypher queries via gRPC. We’ll need to add a few new dependencies to our pom.xml file to pull in the grpc libraries.

   
        <dependency>
            <groupId>io.grpc</groupId>
            <artifactId>grpc-netty</artifactId>
            <version>${grpc.version}</version>
        </dependency>
        <dependency>
            <groupId>io.grpc</groupId>
            <artifactId>grpc-protobuf</artifactId>
            <version>${grpc.version}</version>
        </dependency>
        <dependency>
            <groupId>io.grpc</groupId>
            <artifactId>grpc-stub</artifactId>
            <version>${grpc.version}</version>
        </dependency>

With those in place, let’s start by building our “neo4j.proto” file. Our Service will simply execute a cypher query string and return a cypher query result. Both are just strings, but we could map them properly if we really wanted but we are keeping it simple for now.

   
syntax = "proto3";

option java_multiple_files = true;
option java_package = "com.maxdemarzi";
option java_outer_classname = "Neo4jGRPCProto";

service Neo4jQuery {
    rpc ExecuteQuery (CypherQueryString) returns (stream CypherQueryResult) { }
}

message CypherQueryString {
    string query = 1;
}

message CypherQueryResult {
    string result = 1;
}

Once this file exists we can use maven to run “protobuf:compile” and it will automagically generate some files for us.

I believe there are all the files required for us to build a gRPC Client to talk to our service… which we haven’t created yet. To build our server, we need to use maven again and run “protobuf:compile-custom” which creates a “Neo4jQueryGrpc” file with a Neo4jQueryImplBase class we need to extend. All this magically generated code makes me dizzy or it could be the pain pills.

Anyway, we’ll create a constructor for it passing it the database, which we will call when we register our extension a little later.

   
public class Neo4jGRPCService extends Neo4jQueryGrpc.Neo4jQueryImplBase {
    private static GraphDatabaseService db;

    Neo4jGRPCService(GraphDatabaseService db) {
        Neo4jGRPCService.db = db;
    }

The real work happens in the executeQuery method which begins a transaction, executes the cypher query and streams the results back. Yes that stringObjectMap.toString() is a bit lazy, but just go along with me for now.

 
    @Override
    public void executeQuery(CypherQueryString req, StreamObserver<CypherQueryResult> responseObserver) {
        try (Transaction tx = db.beginTx()) {
            Result result = db.execute(req.getQuery());
            result.stream().forEach(stringObjectMap -> {
                CypherQueryResult r = CypherQueryResult.newBuilder().setResult(stringObjectMap.toString()).build();
                responseObserver.onNext(r);
            });
            tx.success();
        }
        responseObserver.onCompleted();
    }  

Now we need to create a KernelExtensionFactory that registers our gRPC extension and calls that constructor we created earlier.

   
public class RegistergRPCExtensionFactory extends KernelExtensionFactory<RegistergRPCExtensionFactory.Dependencies> {

    @Override
    public Lifecycle newInstance(KernelContext kernelContext, final Dependencies dependencies) throws Throwable {
        return new LifecycleAdapter() {

When we start our Neo4j instance, we will create a gRPC server on port 9999 and add the Neo4jGRPCService we created passing in the database service.

   
            private Server server;

            @Override
            public void start() throws Throwable {
                server  = ServerBuilder.forPort(9999).addService(
                           new Neo4jGRPCService(dependencies.getGraphDatabaseService())).build();
                server.start();
                System.out.println("Started gRPC Server.");
            }

Now let’s see if any of this works by creating a test:

   
public class Neo4jGRPCServiceTest {

    @Rule
    public final Neo4jRule neo4j = new Neo4jRule()
            .withFixture(MODEL_STATEMENT);

We’ll use the @Rule to get a neo4j Service started, and have it start with a single Person node already created with the name of ‘max’. We will query for this node later.

 
private static final String QUERY = "MATCH (n:Person) WHERE n.name = 'max' RETURN n.name";

    private static final String MODEL_STATEMENT =
            "CREATE (n:Person {name:'max'})";  

We’ll also need to create a blocking stub and setup a channel between our test and our server:

  
    private static Neo4jQueryGrpc.Neo4jQueryBlockingStub blockingStub;

    @Before
    public void setup() throws Exception {
        ManagedChannel channel = ManagedChannelBuilder.forAddress("localhost", 9999)
                // Channels are secure by default (via SSL/TLS). For the example we disable TLS to avoid
                // needing certificates.
                .usePlaintext(true)
                .build();
        blockingStub = Neo4jQueryGrpc.newBlockingStub(channel);
    } 

Once all that is in place, we can actually write our test. We create a CypherQueryString using the QUERY we defined earlier. Then call executeQuery from our Stub and check to see if the result matches our expectations. A bunch of these classes were generated for us earlier which makes life easy. One of the benefits of gRPC is that it can generate the client and server files in multiple languages so once you have the .proto file you are off to the races.

  
    @Test
    public void testQuery() throws Exception {

        CypherQueryString queryString = CypherQueryString.newBuilder().setQuery(QUERY).build();
        CypherQueryResult response;

        Iterator<CypherQueryResult> iterator = blockingStub.executeQuery(queryString);
        while (iterator.hasNext()) {
            response = iterator.next();
            Assert.assertEquals("{n.name=max}", response.getResult());
        }
    } 

Which of course it does! … and there you have it. A few lines of code, a couple of hours of reading gRPC documentation and looking wearily at magically autogenerated code and we have a rudimentary gRPC extension on Neo4j. The code as always is on github and if your organization is using gRPC in the back-end and wants to help build a proper gRPC extension, please get in touch and let’s work together on it.

If zeromq and message pack are more your thing, Michael Hunger was messing around with one of those a while back.