Tag Archives: performance

Building a Twitter Clone with Neo4j – Part Two

One of the aspects of my job that I love is the week long proof of concept bootcamps. What it entail is me (or one of my team members) coming onsite to work with your team to build out a POC in just one week. They all vary some what, but I try to stick to a formula that works for me. I spend the first day with the whole team ironing out the Model. This is the trickiest part to get right, because if the model is right, the queries will fall right into place. If the model has to be changed significantly on day 3 let’s say, then a ton of work has to be redone or at least greatly modified. The goal of the end of day one is to have something that looks like the following:
Continue reading

Tagged , , , , , , , ,

Building a Twitter Clone with Neo4j – Part One

Would you believe there is no shortage of Twitter Clone example applications…maybe because they are easy to replicate (ba dum tss, I’ll be here all week.) The earliest one I remember was written by Salvatore Sanfilippo creator of Redis.
It’s a pretty good read, where he explains the basics of Redis (a Key Value store on steroids) and how to model a social network in it. One of the interesting bits to me is how the status updates (tweets) are handled.
Continue reading

Tagged , , , , , , , , ,

Searching for objects using multiple dimensions

Lets take a look at a scenario where you are trying to search for things by their attributes, not their description. They can be users, documents, or any object that could be described by discrete values in multiple dimensions. What does that mean exactly? Well, let me give you an example: searching for a dog. My family includes 2 four legged furry creatures named Tyler and Ronnie. They are my half lab, half golden retrievers. Dogs come in all shapes and sizes, from teacup breeds with adult weights around 5 lbs, to giant Mastiff breeds over 150 lbs. But most people don’t care exactly how much a dog weights, only their general size.


Continue reading

Tagged , , , , , , , , ,

Neo4j is faster than MySQL in performing recursive query

5mysql

A user on StackOverflow was wondering about the performance between Neo4j and MySQL for performing a recursive query. They started with Neo4j performing the query in 240 seconds. Then an optimized cypher query got them down to 40 seconds. Then I got them down to…
Continue reading

Tagged , , , , , , , , , , ,

Our own Multi-Model Database – Part 6

shitty6

Back in Part 2 we ran some JMH tests to see how many empty nodes we could create. Let’s try that test one more time, but adding some properties. Our nodes will have a username, an age and a weight randomly assigned. It’s not a long test, but just enough to give us a ballpark.
Continue reading

Tagged , , , , , , ,

Our own Multi-Model Database – Part 5

shitty5

In part 4 I promised metrics and a shell, so that’s what we’ll tackle today. We are lucky that the Metrics library can be plugged into Jooby without much effort… and double lucky that the Crash library can also be plugged into Jooby without much effort. This is what we are all about here because we’re a bunch of lazy, impatient developers who are ignorant of the limits of our capabilities and who would rather reuse open source code instead of falling victim to the “Not Invented Here” syndrome and do everything from scratch.
Continue reading

Tagged , , , , , , , ,

Our own Multi-Model Database – Part 4

shitty4

Please read parts 1, 2 and 3 before continuing or you’ll be lost.

We started adding an HTTP server to our database last time and created just a couple of end points. Today we’ll finish out the rest of the end points. We’ll also be good open source developers by hooking in Continuous Integration , Test Coverage and Continuous Deployment.

Continue reading

Tagged , , , , , , , ,

Multi-Threading a Traversal

multi-threads

What would you think if I ran out of time,
Would you stand up and walk out on me?
Lend me your eyes and I’ll write you a post
And I’ll try not to run out of memory.

Oh, I get by with a little help from my threads
Mm, I get high with a little help from my threads
Mm, gonna try with a little help from my threads

Today we are going to take a look at how to take a Neo4j traversal and split it up into lots of smaller traversals. I promise it will be electrifying.
Continue reading

Tagged , , , , , , ,

OUR OWN MULTI-MODEL DATABASE – PART 2

shitty2

If you haven’t read part 1 then do that first or this won’t make sense, well nothing makes sense but this specially won’t.

So before going much further I decided to benchmark our new database and found that our addNode speed is phenomenal, but it was taking forever to create relationships. See some JMH benchmarks below:

Benchmark                                                           Mode  Cnt     Score     Error  Units
ChronicleGraphBenchmark.measureCreateEmptyNodes                    thrpt   10  1548.235 ± 556.615  ops/s
ChronicleGraphBenchmark.measureCreateEmptyNodesAndRelationships    thrpt   10     0.165 ±   0.007  ops/s

Each time I was creating 1000 users, so this test shows us we can create over a million empty nodes in one second. Yeah ChronicleMap is damn fast. But then when I tried to create 100 relationships for each user (100,000 total) it was taking forever (about 6 seconds). So I opened up YourKit and you won’t believe what I found out next (come on that’s some good clickbait).
Continue reading

Tagged , , , , , , ,

Our own Multi-Model Database – Part 1

shittydb

I may be remembering this wrong, but I think it was Henry Rollins who once asked, “What came first, the shitty Multi-Model Databases or the Drugs?” His confusion was over whether:

A) there were a bunch of developers dicking around with their Mac laptops and they wrote a shitty database, put it on github, posted on hacker news, and then other developers who were on drugs started using it or…

B) there were a bunch of developers on ketamine and ecstasy and somebody said lets write a shitty database

I think “A” is what probably happens and how we end up with over 300 databases on DB Engines. But what about “B” ? Well I don’t have any good stuff lying around, but I did hurt my foot the other day and the doctors gave me some Tramadol, so lets down some of that and see what happens.
Continue reading

Tagged , , , , , , , ,