nosql | Max De Marzi

Jun 26 2013

Neo4j 2.0 is coming

House Neo4j of Graph Databases is one of the Great Houses of NOSQL and the principal noble house of The Graph; many lesser houses are sworn to them. In days of old they ruled as Kings of the Graph; since the Aggregate Store Conquest they have been Wardens of the Path. Their seat, San Mateo, is an ancient castle renowned for its sushi. Their sigil is a octopus racing across a field of white, and their words are “Neo4j 2.0 Is Coming,” one of only a few house mottoes to be a warning rather than a boast. Members of the family tend to be lean of build and long of face, with golden hair and blue eyes.

Continue reading →

Tagged cypher, graph, graph database, graph databases, neo4j, node labels, nosql

Mar 25 2013

1 Comment

Java, Problems

Permission Resolution with Neo4j – Part 3

Let’s add a couple of performance tests to the mix. We learned about Gatling in a previous blog post, we’re going to use it here again. The first test will randomly choose users and documents (from the graph we created in part 2) and write the results to a file, the second test will re-use the results of the first one and run consistently so we can change hardware, change Neo4j parameters, tune the JVM, etc. and see how they affect our performance.

The full code for the Random Permissions test is here, I’ll just highlight the main parts:
Continue reading →

Tagged cypher, github, graph, graph database, neo4j, nosql, performance, testing

Mar 18 2013

8 Comments

Java, Problems

Permission Resolution with Neo4j – Part 1

People produce a lot of content. Messages, text files, spreadsheets, presentations, reports, financials, etc, the list goes on. Usually organizations want to have a repository of all this content centralized somewhere (just in case a laptop breaks, gets lost or stolen for example). This leads to some kind of grouping and permission structure. You don’t want employees seeing each other’s HR records, unless they work for HR, same for Payroll, or unreleased quarterly numbers, etc. As this data grows it no longer becomes easy to simply navigate and a search engine is required to make sense of it all.

But what if your search engine returns 1000 results for a query and the user doing the search is supposed to only have access to see 4 things? How do you handle this? Check the user permissions on each file realtime? Slow. Pre-calculate all document permissions for a user on login? Slow and what if new documents are created or permissions change between logins? Does the system scale at 1M documents, 10M documents, 100M documents?
Continue reading →

Tagged graph database, java, neo4j, nosql, permissions, relationship graph

Feb 14 2013

6 Comments

Cypher, Deployment, Random

Neo4j and Gatling sitting in a tree, Performance T-E-S-T-ing

neo4j_loves_gatling

I was introduced to the open-source performance testing tool Gatling a few months ago by Dustin Barnes and fell in love with it. It has an easy to use DSL, and even though I don’t know a lick of Scala, I was able to figure out how to use it. It creates pretty awesome graphics and takes care of a lot of work for you behind the scenes. They have great documentation and a pretty active google group where newbies and questions are welcomed.

It ships with Scala, so all you need to do is create your tests and use a command line to execute it. I’ll show you how to do a few basic things, like test that you have everything working, then we’ll create nodes and relationships, and then query those nodes.
Continue reading →

Tagged cypher, gatling, graph database, neo4j, nosql, performance, scala, testing

Nov 27 2012

1 Comment

Java

Pathfinding with Neo4j Unmanaged Extensions

In Extending Neo4j I showed you how to create an unmanaged extension to warm up the node and relationship caches. Let’s try doing something more interesting like exposing the A* (A Star) search algorithm through the REST API. The graph we created earlier looks like this:
Continue reading →

Tagged github, graph database, java, neo4j, nosql

Nov 26 2012

19 Comments

Cypher, Java

Extending Neo4j

One of the great things about Neo4j is how easy it is to extend it. You can extend Neo4j with Plugins and Unmanaged Extensions. Two great examples of plugins are the Gremlin Plugin (which lets you use the Gremlin library with Neo4j) and the Spatial Plugin (which lets you perform spatial operations like searching for data within specified regions or within a specified distance of a point of interest).

Plugins are meant to extend the capabilities of the database, nodes, or relationships. Unmanaged extensions are meant to let you do anything you want. This great power comes with great responsibility, so be careful what you do here. David Montag cooked up an unmanaged extension template for us to use on github so lets give it a whirl. We are going to clone the project, compile it, download Neo4j, configure Neo4j to use the extension, test the extension and tweak it a bit.
Continue reading →

Tagged cypher, extension template, github, graph database, java, mvn, neo4j, nosql, software, spatial operations, technology

Aug 13 2012

6 Comments

Random

Neo4j Internals

An overview of Neo4j Internals

View more presentations from Tobias Lindaaker

It is interesting to see how node, relationship and property records are stored differently on disk and in the cache.

It is all linked lists of fixed size records on disk. Properties are stored as a linked list of property records, each holding a key and value and pointing to the next property. Each node and relationship references its first property record. The Nodes also reference the first relationship in its relationship chain. Each Relationship references its start and end node. It also references the previous and next relationship record for the start and end node respectively.
Continue reading →

Tagged graph database, neo4j, nosql

Aug 10 2012

11 Comments

Problems, Random

Summarize Opinions with a Graph – Part 1

How does the saying go? Opinions are like bellybuttons, everybody’s got one? So let’s say you have an opinion that NOSQL is not for you. Maybe you read my blog and think this Graph Database stuff is great for recommendation engines and path finding and maybe some other stuff, but you got really hard problems and it can’t help you.

I am going to try to show you that a graph database can help you solve your really hard problems if you can frame your problem in terms of a graph. Did I say “you”? I meant anybody, specially Ph.D. students. One trick is to search for “graph based approach to” and your problem.
Continue reading →

Tagged graph, nosql, word graph

Apr 12 2012

10 Comments

Cypher, Heroku, Visualization

Using Sigma.js with Neo4j

I’ve done a few posts recently using D3.js and now I want to show you how to use two other great Javascript libraries to visualize your graphs. We’ll start with Sigma.js and soon I’ll do another post with Three.js.
Continue reading →