Tag Archives: testing

Caching Immutable Id lookups in Neo4j

GiveMeTheCache

If you’ve been following my blog for a while, you probably know I like using YourKit and Gatling for testing end to end requests in Neo4j. Today however we are going to do something a little different. We are going to be micro-benchmarking a very small piece of code within our Unmanaged Extension using a Java library called JMH.

Continue reading

Tagged , , , ,

Scaling Concurrent Writes in Neo4j

concurrent writes

A while ago, I showed you a way to scale Neo4j writes using RabbitMQ. Which was kinda cool, but some of you asked me for a different solution that didn’t involve adding yet another software component to the stack.

Turns out we can do this in just Neo4j using a little help from the Guava library. The solution involved a background service running that holds the writes in a queue, and every once in a while (like say every second) commits those writes in one transaction.
Continue reading

Tagged , , , , , , , ,

Kickstarting a Neo4j Video Series

Learn how to build high performance @neo4j applications with this video training course.

I’m on Kickstarter to ask for your help in order to create a set of videos to teach you how to build high performance Neo4j applications. I am going to capture the lessons I’ve learned over the past 4 years working with graph databases and share them with you.

These videos will teach you everything you need to know about building high performance applications using Neo4j.
Continue reading

Tagged , , , , , ,

Translating Cypher To Neo4j Java API 2.0

cypher-translate-2.0ish600x293

About 6 months ago we looked at how to translate a few lines of Cypher in to way too much Java code in version 1.9.x. Since then Cypher has changed and I suck a little less at Java, so I wanted to share a few different ways to translate one into the other just in case you stuck in a mid-eighties time warp and are paid by the number of lines of code you write per hour.

But first, lemme take a #Selfie let’s make some data. Michael Hunger has a series of blog posts on getting and creating data in Neo4j, we’ll steal borrow his ideas. Let’s create 100k nodes:

WITH ["Jennifer","Michelle","Tanya","Julie","Christie","Sophie","Amanda","Khloe","Sarah","Kaylee"] AS names 
FOREACH (r IN range(0,100000) | CREATE (:User {username:names[r % size(names)]+r}))

Continue reading

Tagged , , , , , , , , , , ,

It’s over 9000! Neo4j on WebSockets

it__s_over_9000_

In the last blog post we managed to run Neo4j at Ludicrous Speed over http using Undertow and get to about 8000 requests per second. If we needed more speed we can scale up the server or we can scale out to multiple servers by switching out the GraphDatabaseFactory and using the HighlyAvailableGraphDatabaseFactory class instead in Neo4j Enterprise Edition.

But can we go faster on a single server without new hardware? Well… yes, if we’re willing to drop http and switch to Web Sockets.

Continue reading

Tagged , , , , , , , ,

The Power of Open Source Software

opensource-400

One of the benefits of Open Source Software is that if you want to change how something is done, you can. At Neo Technology, we have a small team of “Field Engineers” who don’t really work ON the product but rather WITH the product. We help our customers with issues of all kinds, answer questions, give suggestions and whatever we need to do to make people’s project successful. A little while back I had a support ticket for a traversal that was taking longer than they hoped it would.

Think about a social network, one of the things you may want to do is tell the user how big their friends network is. But why stop there? How about their friends of friends or even friends of friends of friends network? These are the kind of questions graph databases excel at compared to relational databases. Let’s take a look at what they were doing:
Continue reading

Tagged , , , , , ,

Scaling Up

scaling-up

Rock climbing is a physically and mentally demanding sport, it test the limits of one’s strength, endurance, agility, balance and concentration. Sasha DiGiulian is one of the best rock climbers in the world. I can’t get past 15 feet without starting to panic and freak out. Maybe it’s because I’m afraid of heights…and overweight, but I’m just not right for that kind of challenge.
Continue reading

Tagged , , , , ,

Scaling Writes

scaling_writes

Most of the applications using Neo4j are read heavy and scale by getting more powerful servers or adding additional instances to the HA cluster. Writes however can be a little bit tricker. Before embarking on any of the following strategies it is best that the server is tuned. See the Linux Performance Guide for details. One strategy we’ve seen already is splitting the reads and writes to the cluster, so the writes only go to the Master. The brave can even change the push factor to zero and set a pull interval only in neo4j/conf/neo4j.properties:
Continue reading

Tagged , , , , ,

Permission Resolution with Neo4j – Part 3

write_automated_test2

Let’s add a couple of performance tests to the mix. We learned about Gatling in a previous blog post, we’re going to use it here again. The first test will randomly choose users and documents (from the graph we created in part 2) and write the results to a file, the second test will re-use the results of the first one and run consistently so we can change hardware, change Neo4j parameters, tune the JVM, etc. and see how they affect our performance.

The full code for the Random Permissions test is here, I’ll just highlight the main parts:
Continue reading

Tagged , , , , , , ,

Neo4j and Gatling sitting in a tree, Performance T-E-S-T-ing

neo4j_loves_gatling

I was introduced to the open-source performance testing tool Gatling a few months ago by Dustin Barnes and fell in love with it. It has an easy to use DSL, and even though I don’t know a lick of Scala, I was able to figure out how to use it. It creates pretty awesome graphics and takes care of a lot of work for you behind the scenes. They have great documentation and a pretty active google group where newbies and questions are welcomed.

It ships with Scala, so all you need to do is create your tests and use a command line to execute it. I’ll show you how to do a few basic things, like test that you have everything working, then we’ll create nodes and relationships, and then query those nodes.
Continue reading

Tagged , , , , , , ,