Tag Archives: software

Our own Multi-Model Database – Part 5

shitty5

In part 4 I promised metrics and a shell, so that’s what we’ll tackle today. We are lucky that the Metrics library can be plugged into Jooby without much effort… and double lucky that the Crash library can also be plugged into Jooby without much effort. This is what we are all about here because we’re a bunch of lazy, impatient developers who are ignorant of the limits of our capabilities and who would rather reuse open source code instead of falling victim to the “Not Invented Here” syndrome and do everything from scratch.
Continue reading

Tagged , , , , , , , ,

Our own Multi-Model Database – Part 4

shitty4

Please read parts 1, 2 and 3 before continuing or you’ll be lost.

We started adding an HTTP server to our database last time and created just a couple of end points. Today we’ll finish out the rest of the end points. We’ll also be good open source developers by hooking in Continuous Integration , Test Coverage and Continuous Deployment.

Continue reading

Tagged , , , , , , , ,

Our own Multi-Model Database – Part 3

shitty3

If you haven’t read part 1 and part 2 then do that first or you’ll have no clue what I’m doing, and I’d like to be the only one not knowing what I’m doing.

We’ve built the beginnings of this database but so far it’s just a library and for it to be a proper database we need to be able to talk to it. Following the Neo4j footsteps, we will wrap a web server around our database and see how it performs.

There are a ton of Java based frameworks and micro-frameworks out there. Not as bad as the Javascript folks, but that still leaves us with a lot of choices. So as any developer would do I turn to benchmarks done by other people of stuff that doesn’t apply to me, and you won’t believe what I found –scratch that, yes you will, I got benchmarks.
Continue reading

Tagged , , , , , , ,

OUR OWN MULTI-MODEL DATABASE – PART 2

shitty2

If you haven’t read part 1 then do that first or this won’t make sense, well nothing makes sense but this specially won’t.

So before going much further I decided to benchmark our new database and found that our addNode speed is phenomenal, but it was taking forever to create relationships. See some JMH benchmarks below:

Benchmark                                                           Mode  Cnt     Score     Error  Units
ChronicleGraphBenchmark.measureCreateEmptyNodes                    thrpt   10  1548.235 ± 556.615  ops/s
ChronicleGraphBenchmark.measureCreateEmptyNodesAndRelationships    thrpt   10     0.165 ±   0.007  ops/s

Each time I was creating 1000 users, so this test shows us we can create over a million empty nodes in one second. Yeah ChronicleMap is damn fast. But then when I tried to create 100 relationships for each user (100,000 total) it was taking forever (about 6 seconds). So I opened up YourKit and you won’t believe what I found out next (come on that’s some good clickbait).
Continue reading

Tagged , , , , , , ,

Connected

connected

The Stereo MC’s song “Connected” could be about some recently gained insight and the realization that maybe some of the people you held dear are phonies and while the reality of the situation is scary, you cannot allow yourself to turn a blind eye anymore or allow yourself to backslide by disconnecting from the real world.

Or it could be a warning about how we’ve all been blinded by SQL databases for too long and we must instead look to connect our data with Graph Databases. About how those new connections may be scary (like because of fraud detection) but they are necessary to better understand reality.

Either way, we may want to see if two nodes in Neo4j are connected and I’m going to show you how to do that faster.
Continue reading

Tagged , , , , , , , , ,

Delivering a Graph Based Search solution to slightly wrong data

oops

When it comes to databases, having good clean data is always important. More so with Graphs which deal with concepts as nodes and their relationships between them. Inevitably, you will run into messy data and have to deal with it. In a lot of the projects our customers work on they are dealing with connecting multiple data sources to get to a “golden record” or single source of truth. A lofty goal, sometimes impossible to achieve, but we can use the relationships of the data to help us come close.

One option is to extract the features (or tags) of a composite object and see if any other object shares most of these features. If that is the case then they are possibly the same object and should be merged instead of creating a new record. A partial subgraph match is something akin to a recommendation engine in Neo4j and pretty trivial to write. Take a look back at a few old blog posts for ideas.
Continue reading

Tagged , , , , , , , , , ,

Bidirectional Traversals in Space

firefly

If you have never watched Firefly, then stop whatever you are doing and get to it, you can come back and read this post later. Ok good, now where were we. Firefly. The series is set a few hundred years from now, after people begin to terraform a new star system and it follows the adventures of the renegade crew of Serenity, a “Firefly-class” spaceship whose work consists of cargo runs or smuggling while failing to stay out of trouble. There is no faster than light travel in this series, so ships can’t just “warp” where ever they want. Instead they travel about from planets and moons, exchanging cargo, refueling and trying to make a living. We are going to model “The Verse” of Firefly in Neo4j, and see how we can find routes to move our illicit cargo from one place to another.
Continue reading

Tagged , , , , , , , , ,

Flight Search with the Neo4j Traversal API

Screen Shot 2015-08-30 at 2.21.07 AM

Before Cypher came along, if you wanted to describe a graph traversal in Neo4j you would use the Traversal Framework Java API. The Traversal API is one of the many hidden gems of Neo4j and today we are going to take a closer look at it. Traversing a graph is about going on a journey. All journeys have a starting point (or points) so that’s the first thing we have to do, figure out where in the graph we begin. It can be a single node, or multiple ones, but they will go on the journey following the same rules, so its easier if it’s just one node or nodes of the same “type”.
Continue reading

Tagged , , , , , , , , ,

Importing the Hacker News Interest Graph

HackerNews-799e9e47

Graphs are everywhere. Think about the computer networks that allow you to read this sentence, the road or train networks that get you to work, the social network that surrounds you and the interest graph that holds your attention. Everywhere you look, graphs. If you manage to look somewhere and you don’t see a graph, then you may be looking at an opportunity to build one. Today we are going to do just that. We are going to make use of the new Neo4j Import tool to build a graph of the things that interest Hacker News.
Continue reading

Tagged , , , , , , , , , , , ,

One Direction Relationships in Neo4j

onedirectionchop

In the Neo4j Property Graph model, every single Relationship must be Typed and Directed. This means they must have a specific name (FRIENDS, LIKES, FOLLOWS, etc) and have a Start Node and an End Node to show direction. What’s neat is that when you write your queries you can choose to ignore that. The following queries are all valid:

// Get all the people I follow 
MATCH (u1:Person)-[:FOLLOWS]->(u2:Person)
WHERE u1.username = "maxdemarzi"
RETURN u2.username

// Get all the people that I follow or follow me
MATCH (u1:Person)-[:FOLLOWS]-(u2:Person)
WHERE u1.username = "maxdemarzi"
RETURN u2.username

// Get all the people related to me 
MATCH (u1:Person)--(u2:Person)
WHERE u1.username = "maxdemarzi"
RETURN u2.username

Continue reading

Tagged , , , , ,