Problems | Max De Marzi

Feb 18 2016

Scaling Cypher Writes

salt-pepa-writes

Let’s talk about writes, baby. Let’s talk about you and me. Let’s talk about all the good things. And the bad things that may be. Let’s talk about writes, and indexing and batching, and transactions in Neo4j. Let’s start with my environment. A 3 year old MacBook Pro (dying to get the new ones… once they finally come out) running a 4 core 2.3 GHz Intel Core i7 that is hyper-threading and pretending to have 8. An Apple SM256E SSD that is about average as far as SSDs go. So definitely not a production grade server, so bear that in mind.
Continue reading →

Tagged cypher, graph database, java, neo4j, performance, technology, testing

Nov 20 2015

1 Comment

Java, Problems

Bidirectional Traversals in Space

If you have never watched Firefly, then stop whatever you are doing and get to it, you can come back and read this post later. Ok good, now where were we. Firefly. The series is set a few hundred years from now, after people begin to terraform a new star system and it follows the adventures of the renegade crew of Serenity, a “Firefly-class” spaceship whose work consists of cargo runs or smuggling while failing to stay out of trouble. There is no faster than light travel in this series, so ships can’t just “warp” where ever they want. Instead they travel about from planets and moons, exchanging cargo, refueling and trying to make a living. We are going to model “The Verse” of Firefly in Neo4j, and see how we can find routes to move our illicit cargo from one place to another.
Continue reading →

Tagged firefly, github, graph database, java, neo4j, network, software, space, technology, transportation

Oct 16 2015

1 Comment

Java, Problems, Random, Testing

Benchmarks and Superchargers

For the most part, I hate competitive benchmarks. The vendor who publishes them always seems to come out on top regardless. The numbers are always amazing, but once you start digging in a little bit you start to see faults in what is actually being measured and it never applies to real world workloads. For example you have Cassandra claiming 1 Million writes per second on 300 servers. Then Aerospike claiming 1 Million writes per second on 50 servers. MongoDB claiming almost 32k writes per second on a single server, but claiming Cassandra can only do 6k w/s and Couch can only do 1.2k w/s on a single server… Then ScyllaDB has almost 2 Million writes per second on 3 servers blowing everybody away.
Continue reading →

Tagged cypher, graph, graph database, java, neo4j, performance, testing

Sep 04 2015

7 Comments

Java, Problems

Flight Search with the Neo4j Traversal API

Before Cypher came along, if you wanted to describe a graph traversal in Neo4j you would use the Traversal Framework Java API. The Traversal API is one of the many hidden gems of Neo4j and today we are going to take a closer look at it. Traversing a graph is about going on a journey. All journeys have a starting point (or points) so that’s the first thing we have to do, figure out where in the graph we begin. It can be a single node, or multiple ones, but they will go on the journey following the same rules, so its easier if it’s just one node or nodes of the same “type”.
Continue reading →

Tagged cypher, github, graph database, java, neo4j, network, nosql, relationship, software, technology

Aug 26 2015

14 Comments

Problems, Random

Modeling Airline Flights in Neo4j

Actor Leonardo DiCaprio as Frank Abagnale in the Steven Spielberg movie “Catch Me If You Can”

If you’ve come to any of the Neo4j Data Modeling classes I’ve taught, you’ve must have heard me say “your model depends on both your data and your queries” about a million times. Let us take a closer dive into what this means by looking at how one might model airline flight data in Neo4j.
Continue reading →

Tagged graph, graph database, modeling, models, neo4j

Apr 14 2015

2 Comments

Cypher, Problems, Random

Importing the Hacker News Interest Graph

Graphs are everywhere. Think about the computer networks that allow you to read this sentence, the road or train networks that get you to work, the social network that surrounds you and the interest graph that holds your attention. Everywhere you look, graphs. If you manage to look somewhere and you don’t see a graph, then you may be looking at an opportunity to build one. Today we are going to do just that. We are going to make use of the new Neo4j Import tool to build a graph of the things that interest Hacker News.
Continue reading →

Tagged cypher, graph database, graph databases, hacker news, hcir, interest graph, neo4j, network, nosql, relationship graph, ruby, software, technology

Mar 25 2015

5 Comments

Cypher, Java, Problems

Triggers in Neo4j

One of the often overlooked features in Neo4j is the “TransactionEventHandler” capabilities… better known in the database world as “Triggers“. When a transaction occurs, we can analyze that event and decide to take some action. To accomplish this, we’ll write a “Kernel Extension” ( a little different from the Unmanaged Extensions we’ve seen on this blog ) to tie in our trigger.

Continue reading →

Tagged graph database, java, neo4j, network, nosql, relationship graph, triggers

Mar 17 2015

6 Comments

Cypher, Java, Problems

One Direction Relationships in Neo4j

In the Neo4j Property Graph model, every single Relationship must be Typed and Directed. This means they must have a specific name (FRIENDS, LIKES, FOLLOWS, etc) and have a Start Node and an End Node to show direction. What’s neat is that when you write your queries you can choose to ignore that. The following queries are all valid:

// Get all the people I follow 
MATCH (u1:Person)-[:FOLLOWS]->(u2:Person)
WHERE u1.username = "maxdemarzi"
RETURN u2.username

// Get all the people that I follow or follow me
MATCH (u1:Person)-[:FOLLOWS]-(u2:Person)
WHERE u1.username = "maxdemarzi"
RETURN u2.username

// Get all the people related to me 
MATCH (u1:Person)--(u2:Person)
WHERE u1.username = "maxdemarzi"
RETURN u2.username

Continue reading →

Tagged graph database, java, neo4j, performance, software, technology

Feb 27 2015

Caching Immutable Id lookups in Neo4j

If you’ve been following my blog for a while, you probably know I like using YourKit and Gatling for testing end to end requests in Neo4j. Today however we are going to do something a little different. We are going to be micro-benchmarking a very small piece of code within our Unmanaged Extension using a Java library called JMH.

Continue reading →

Tagged cache, graph database, neo4j, performance, testing

Feb 24 2015

Remote Profiling Neo4j with YourKit on AWS

A few months ago, Mark Needham blogged about how to setup remote monitoring of Neo4j using YourKit. I was asked the other day about getting a few more details on how to do this on Amazon, so here is my attempt at that. The first thing we’ll do is setup Neo4j on a Virtual Private Cloud. It’s good practice to not put your databases directly on the public internet.
Continue reading →

Tagged graph database, neo4j, performance, profiling

Max De Marzi

Graphs, Graphs, and nothing but the Graphs

Category Archives: Problems

Scaling Cypher Writes

Bidirectional Traversals in Space

Benchmarks and Superchargers

Flight Search with the Neo4j Traversal API

Modeling Airline Flights in Neo4j

Importing the Hacker News Interest Graph

Triggers in Neo4j

One Direction Relationships in Neo4j

Caching Immutable Id lookups in Neo4j

Remote Profiling Neo4j with YourKit on AWS