Category Archives: Problems

Mutual Fund Benchmarks with Neo4j

Just the other day I had a conversation with an Investment Risk Manager about one of the data problems his team was working on and he was wondering if Neo4j could help. Imagine you have about 20,000 mutual funds and etfs and you want to track how they measure up against a benchmark like say the returns of the S&P 500. I’m sorry did I say one? I meant all of them, let’s say 2,000 different benchmarks… and you want to track it every day, for a rolling 5 years period. So that’s 20,000 securities * 2000 benchmarks * 5 years * 252 trading days a year (on average)… or 50 billion data points. That’s a BIG join table if we were using a relational database. How can we efficiently model this in Neo4j?
Continue reading

Tagged , , , , , , , ,

Bill of Materials in Neo4j

Where is da BOM? The above question asks, and the obvious answer is right in the middle of your organization. Nestled between Manufacturing, Design, Sales and Supply Chain. But I have a better answer. Your Bill of Materials should be in Neo4j. Today, I’ll show you why.
Continue reading

Tagged , , , , , , , , , ,

Counting Nodes with Multiple Labels

We have over 6000 users in our #neo4j-users slack channel and get all kinds of requests. About a month ago Thomas Shields asked:

Should counting the set of things with 2 labels really take so long? I’ve got 48M nodes with LabelA and LabelB and the query `MATCH (n:LabelA:LabelB) RETURN COUNT(n)` is taking 80-90 seconds

Let’s see what’s going on by creating a small version of his graph. We will create 1M nodes of LabelA, then 1M nodes with both LabelA and LabelB, and then 1M nodes with just Label B:
Continue reading

Tagged , , , , , ,

Work Order Management with Neo4j

I look terrible in a bikini (take my word for it) but I’d love me a Lamborghini. However, in order to afford nice things, we need to do as the song says and get to work…and we need to manage and prioritize that work somehow. Today, I’m going to show you how to build part of a work order management system with Neo4j.

I’m going to build an evented work order model. So let’s say our Order gets created, then based on what it is, pieces of Work need to happen. This work is performed by some Provider (whether internal or external) and that work can be broken down into Tasks that have dependencies on Events that have occurred. How would this look like in the graph? Glad you asked:
Continue reading

Tagged , , , , , , , , , , ,

Building a Boolean Logic Rules Engine in Neo4j

A boolean logic rules engine was the first project I did for Neo4j before I joined the company some 5 years ago. I was working for some start-up at the time but took a week off to play consultant. I had never built a rules engine before, but as far as I know ignorance has never stopped anyone from trying. Neo4j shipped me to the client site, and put me in a room with a projector and a white board where I live coded with an audience of developers staring at me, analyzing every keystroke and cringing at every typo and failed unit test. I forgot what sleep was, but managed to figure it out and I lost all sense of fear after that experience.

The data model chained together fact nodes with criss crossing relationships each chain containing the same path id property we followed until reaching an end node which triggered a rule. There were a few complications along the way and more complexity near the end for ordering and partial matches. The traversal ended up being some 40 lines of the craziest Gremlin code I ever wrote, but it worked. After the proof of concept, the project was rewritten using the Neo4j Java API because at the time only a handful of people could look at a 40 line Gremlin script and not shudder in horror. I think we’re up to two handfuls now.
Continue reading

Tagged , , , , , , , , , ,

Finding Triplets with Neo4j

A user had an interesting Neo4j question on Stack Overflow the other day:

I have two types of nodes in my graph. One type is Testplan and the other is Tag. Testplans are tagged to Tags. I want most common pairs of Tags that share the same Testplans with a Tag having a specific name. I have been able to achieve the most common Tags sharing the same Testplan with one Tag, but getting confused when trying to do it for pairs of Tags.

Continue reading

Tagged , , , , , , , ,

Using a Cuckoo Filter for Unique Relationships

We often see a pattern in Neo4j applications where a user wants to create one and only one relationship between two nodes. For example a User follows another User on a social network. We don’t want to accidentally create a second follows relationship because that may create errors such as duplicate entries on their feed, or errors unfollowing or blocking them, or even skew recommendation algorithms. Also it is just plain wasteful, and while an occasional duplicate relationship won’t be a big deal, millions of them could.

So how do we deal with this?
Continue reading

Tagged , , , , , , , , , , , , ,

Flight Search with Neo4j

I think I am going to take the opportunity to explain why I love graphs in this blog post. I’m going to try to explain why looking at problems from the graph point of view opens you up to creative solutions and makes back-end development fun again. The context of our post is flight search, but our true mission is to figure out how to traverse a graph quickly and efficiently so we can apply our knowledge to other problems.

A long while back, I showed you different ways to model airline flight data. When it comes to modeling in graphs, the lesson to take away is that there is no right way. The optimal model is heavily dependent on the queries you want to ask. Just to prove the point, I’m going to show you yet another way to model the airline flight data that is truly optimized for flight search. If you recall, our last model looked like:
Continue reading

Tagged , , , , , , , , , , ,

Building a Twitter Clone with Neo4j – Part Four

We left off last time having just added the ability to follow people, see who we’ve followed and has followed us, block and unblock people and finally see whom we have put on our naughty list of blocked users. So we have a social network where people can create relationships, but they have nothing to say because we haven’t implemented that yet!
Continue reading

Tagged , , , , , , , ,

Building a Twitter Clone with Neo4j – Part Three

In part two we defined our API and got registering a user, checking a user and getting a user profile. A social network of unconnected people doesn’t live up to its name, so let’s go ahead and build the ability to follow people.

Continue reading

Tagged , , , , , , , ,