A Peek behind the Neo4j Lucene Index Curtain

wizard-of-oz

Did you know you can write Javascript in the Neo4j console to access the Neo4j API?
Try it. Open up your Neo4j Web Admin Console and type:

neo4j-sh (0)$ eval db
EmbeddedGraphDatabase [data/graph.db]

OMG! I know, Neo4j is crazy. So much to play with, I’ve been at it for a few years and I haven’t even dug into this area. What else can we do here?


neo4j-sh (0)$ eval db.config
{ephemeral=false, keep_logical_logs=true, neo4j.ext.udc.source=server, neostore.nodestore.db.mapped_memory=368M, neostore.propertystore.db.arrays.mapped_memory=1277M, neostore.propertystore.db.mapped_memory=1326M, neostore.propertystore.db.strings.mapped_memory=1127M, neostore.relationshipstore.db.mapped_memory=1624M, node_auto_indexing=true, node_keys_indexable=name,title,released,tagline, remote_shell_enabled=true, store_dir=data/graph.db}

Pretty cool right? I have a Matrix movie graph loaded, let’s try to pull up a node by index:

eval db.index().forNodes("node_auto_index").get("title","The Matrix").getSingle().getProperty("title").toString();
The Matrix

You can go wild here, but what I really wanted to do is take a peek behind the API curtain and see what’s really going on with the Neo4j Indexes.

Screen Shot 2013-03-15 at 12.31.49 PM

Luke is a handy development and diagnostic tool, which accesses already existing Lucene indexes and allows you to display and modify their content in several ways:

  • browse by document number, or by term
  • view documents / copy to clipboard
  • retrieve a ranked list of most frequent terms
  • execute a search, and browse the results
  • analyze search results
  • selectively delete documents from the index
  • reconstruct the original document fields, edit them and re-insert to the index
  • optimize indexes
  • and much more…

You can download the Luke 4.0.0-ALPHA standalone binary and run it with:

java -jar lukeall-4.0.0-ALPHA.jar

Then point it to one of your Neo4j Indexes (here I am using the node_auto_index since I have configured my Neo4j instance to use it).

/neo4j/data/graph.db/index/lucene/node/node_auto_index/segments.gen

If your database is running, you will want to open it in Read-Only mode.

Here you can see the fields that are indexed as well as some statistics about them:

Luke-screenshot1

You browse through each record in the index and can even query the syntax directly and see what’s inside:

Luke screenshot 2

Take a look at this blog post for a more detailed look at Luke’s capabilities.

Be careful what you do here, and I think it’s better to play with a copy of your database instead of making any changes this way to the live database.

Tagged , , , , ,

One thought on “A Peek behind the Neo4j Lucene Index Curtain

  1. [...] A Peek Behind the Neo4j Lucene Index Curtain by Max De Marzi. [...]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 1,678 other followers

%d bloggers like this: