Max De Marzi

One of the great things about Neo4j is how easy it is to extend it. You can extend Neo4j with Plugins and Unmanaged Extensions. Two great examples of plugins are the Gremlin Plugin (which lets you use the Gremlin library with Neo4j) and the Spatial Plugin (which lets you perform spatial operations like searching for data within specified regions or within a specified distance of a point of interest).

Plugins are meant to extend the capabilities of the database, nodes, or relationships. Unmanaged extensions are meant to let you do anything you want. This great power comes with great responsibility, so be careful what you do here. David Montag cooked up an unmanaged extension template for us to use on github so lets give it a whirl. We are going to clone the project, compile it, download Neo4j, configure Neo4j to use the extension, test the extension and tweak it a bit.

Start by cloning the project:

git clone git://github.com/dmontag/neo4j-unmanaged-extension-template.git
cd neo4j-unmanaged-extension-template

Let’s add Neography to our gemfile and install it:

echo "gem 'neography' " > Gemfile
bundle install 

Then let’s go ahead and install Neo4j. For this project I am going to use Neo4j 1.8 enterprise edition:

echo "require 'neography/tasks'" > Rakefile
rake neo4j:install[enterprise,1.8]

Make sure you are not running Neo4j already as the next step will download and build the unmanaged extension template which includes some tests that will fail if you already have Neo4j running somewhere.

mvn clean package

This will download and do its thing unless you don’t have Maven installed (type mvn -version to see if you do). Finally you should see:

...
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ unmanaged-extension-template ---
[INFO] Building jar: /Users/maxdemarzi/Projects/neo4j-unmanaged-extension-template/target/unmanaged-extension-template-1.0.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7.153s
[INFO] Finished at: Tue Nov 13 22:16:40 CST 2012
[INFO] Final Memory: 27M/218M
[INFO] ------------------------------------------------------------------------

Now we can copy the jar file it created to our Neo4j plugins directory:

cp target/unmanaged-extension-template-1.0.jar neo4j/plugins/

Before this will work, we need to tell Neo4j about our new unmanaged extension by adding a line to the neo4j-server.properties file in the neo4j/conf directory:

echo "org.neo4j.server.thirdparty_jaxrs_classes=org.neo4j.example.unmanagedextension=/example" >> neo4j/conf/neo4j-server.properties

We are telling Neo4j to load this unmanaged extension and make “/example” the rest api end-point. Now we can start the Neo4j server…

rake neo4j:start

… and test this out using curl:

curl http://localhost:7474/example/service/helloworld
Hello World!

or via the http console in the web admin:

The reason this works is because the jar file contains the contents of /src/main/java/org/neo4j/example/unmanagedextension/MyService.java in which you will find:

@Path("/service")
public class MyService {

    @GET
    @Path("/helloworld")
    public String helloWorld() {
        return "Hello World!";
    }

This very simple method just returns a string from the server whenever we call it. Let’s try something a little more useful, like warming up the Node and Relationship Caches. We need to first create some nodes and relationships. Edit your Rakefile to look like this:

require 'neography'
require 'neography/tasks'

namespace :neo4j do
  task :create do
    neo = Neography::Rest.new
    nodes = {}
    5.times do |x|
      5.times do |y|
       node = neo.create_node({:x => x.to_f, :y=> y.to_f}) 
       nodes["#{x}-#{y}"] = node["self"].split('/').last
      end
    end
    
    4.times do |x|
      4.times do |y|
       neo.create_relationship("next_to", 
                               nodes["#{x}-#{y}"], 
                               nodes["#{x+1}-#{y}"], 
                               {:time => (1 + rand(4)).to_f }) 
       neo.create_relationship("next_to", 
                               nodes["#{x}-#{y}"], 
                               nodes["#{x}-#{y+1}"], 
                               {:time => (1 + rand(4)).to_f }) 
      end
      neo.create_relationship("next_to", 
                              nodes["#{x}-4"], 
                              nodes["#{x+1}-4"], 
                              {:time => (1 + rand(4)).to_f }) 
      neo.create_relationship("next_to", 
                              nodes["4-#{x}"], 
                              nodes["4-#{x+1}"], 
                              {:time => (1 + rand(4)).to_f }) 
    end
        
  end
end

Then run:

rake neo4j:create

… to create your graph. I’ll explain why we created this particular graph in a follow up blog post, but for now back to our MyService.java file. We will be using the GlobalGraphOperations tool and adding the following:

import org.neo4j.tooling.GlobalGraphOperations;
import org.neo4j.graphdb.Node;
import org.neo4j.graphdb.Relationship;

    @GET
    @Path("/warmup")
    public String warmUp(@Context GraphDatabaseService db) {
		Node start;
		for ( Node n : GlobalGraphOperations.at( db ).getAllNodes() ) {
		   n.getPropertyKeys();
		   for ( Relationship relationship : n.getRelationships() ) {
              start = relationship.getStartNode();
           }
        }
        for ( Relationship r : GlobalGraphOperations.at( db ).getAllRelationships() ) {
          r.getPropertyKeys();
          start = r.getStartNode();
        }
    return "Warmed up and ready to go!";
    }

The code above first iterates through every node, brings up its properties, then calls up the relationships that node has. The code then iterates through every relationship, brings up its properties, then calls up the nodes it connects. The sequence above effectively warms up your caches. If your graph fits in memory, this will prime the database. If your graph doesn’t fit in memory maybe you want to specify a certain set of nodes or relationships to warm up. Either way you probably don’t want to run this over and over again or expose it to your end users. Remember what I said about being responsible with what you do here.

You’ll have to stop Neo4j, recompile it, copy the new jar file over the existing one, restart Neo4j, and run the command.

rake neo4j:stop
mvn clean package
cp target/unmanaged-extension-template-1.0.jar neo4j/plugins/
rake neo4j:start
curl http://localhost:7474/example/service/warmup

If everything went well you should see:

Warmed up and ready to go!

…and looking at your node and relationship caches in the Neo4j web admin server info page:

If you don’t see these you are probably running the community edition instead of the enterprise edition. If the numbers are all zero then something went wrong. Retrace your steps and let me know if any part of this is confusing so I can do a better job explaining it. You can see my copy of MyService.java on github.

By the way, if you just want a warm cache you can do the same with Cypher by using:

START n=node(*) 
MATCH n -[r?]-> () 
RETURN count(n.property_i_do_not_have?)+count(r.property_i_do_not_have?)

Did you know that “Cypher” is a term used to describe freestyle rap in a group setting?

Well, now you know. Stay tuned for part two of this series.