Scaling Writes

scaling_writes

Most of the applications using Neo4j are read heavy and scale by getting more powerful servers or adding additional instances to the HA cluster. Writes however can be a little bit tricker. Before embarking on any of the following strategies it is best that the server is tuned. See the Linux Performance Guide for details. One strategy we’ve seen already is splitting the reads and writes to the cluster, so the writes only go to the Master. The brave can even change the push factor to zero and set a pull interval only in neo4j/conf/neo4j.properties:
Continue reading

Tagged , , , , ,

Connected

connected

Connected: The Surprising Power of Our Social Networks and How They Shape Our Lives is a mind bending look at how no matter how individual we think we are, the people around us have a great amount of influence in our lives. One of the authors James Fowler was at GraphConnect 2012 and gave a presentation on this idea:
Continue reading

Tagged , , , , , , ,

The Last Mile

Last-Mile

The “last mile” is a term used in the telecommunications industry that refers to delivering connectivity to the customers that will actually be using the system. In the sense of Graph Databases, it refers to how well the end user can extract value and insight from the graph. We’ve already seen an example of this concept with Graph Search, allowing a user to express their requests in natural language. Today we’ll see another example. We’ll be taking advantage of the features of Neo4j 2.0 to make this work, so be sure to have read the previous post on the matter.

We’re going to be using VisualSearch.js made by Samuel Clay of NewsBlur. VisualSearch.js enhances ordinary search boxes with the ability to autocomplete faceted search queries. It is quite easy to customize and there is an annotated walkthrough of the options available. You can see what it does in the image below, or click it to try their demo.

Continue reading

Tagged , , , , , , ,

Neo4j 2.0 is coming

neoiscoming

House Neo4j of Graph Databases is one of the Great Houses of NOSQL and the principal noble house of The Graph; many lesser houses are sworn to them. In days of old they ruled as Kings of the Graph; since the Aggregate Store Conquest they have been Wardens of the Path. Their seat, San Mateo, is an ancient castle renowned for its sushi. Their sigil is a octopus racing across a field of white, and their words are “Neo4j 2.0 Is Coming,” one of only a few house mottoes to be a warning rather than a boast. Members of the family tend to be lean of build and long of face, with golden hair and blue eyes.

Continue reading

Tagged , , , , , ,

Visualizing the news with Vivagraph.js

neo_news

Today I want to introduce you to VivaGraphJS – a JavaScript Graph Drawing Library made by Andrei Kashcha of Yasiv. It supports rendering graphs using WebGL, SVG or CSS formats and currently supports a force directed layout. The Library provides an API which tracks graph changes and reflect changes on the rendering surface which makes it fantastic for graph exploration.

Today we will be integrating it with Neo4j and the Alchemy API.

Continue reading

Tagged , , , , , , ,

Knowledge Bases in Neo4j

cnet5promo

From the second we are born we are collecting a wealth of knowledge about the world. This knowledge is accumulated and interrelated inside our brains and it represents what we know. If we could export this knowledge and give it to a computer, it would look like ConceptNet. ConceptNet is a semantic network that…

…is built from nodes representing concepts, in the form of words or short phrases of natural language, and labeled relationships between them. These are the kinds of things computers need to know to search for information better, answer questions, and understand people’s goals.

Continue reading

Tagged , , , , , , , , ,

Match Making with Neo4j

groucho_marx

It is better to have loft and lost than to never have loft at all.” — Groucho Marx

In the “Matches are the new Hotness” blog post, I showed how to connect a person to a job via a location and skills. We’re going to look at a variation on the theme today by matching people to other people by what they want in a potential mate. We’re gonna use Neo4j to bring the love.
Continue reading

Permission Resolution with Neo4j – Part 3

write_automated_test2

Let’s add a couple of performance tests to the mix. We learned about Gatling in a previous blog post, we’re going to use it here again. The first test will randomly choose users and documents (from the graph we created in part 2) and write the results to a file, the second test will re-use the results of the first one and run consistently so we can change hardware, change Neo4j parameters, tune the JVM, etc. and see how they affect our performance.

The full code for the Random Permissions test is here, I’ll just highlight the main parts:
Continue reading

Tagged , , , , , , ,

Permission Resolution with Neo4j – Part 2

the-princess-bride-original3

Let’s try tackling something a little bigger. In Part 1 we created a small graph to test our permission resolution graph algorithm and it worked like a charm on our dozen or so nodes and edges. I don’t have fast hands, so instead of typing out a million node graph, we’ll build a graph generator and use the batch importer to load it into Neo4j. What I want to create is a set of files to feed to the batch-importer.
Continue reading

Tagged , , , , ,

Permission Resolution with Neo4j – Part 1

i_can_haz_permissions

People produce a lot of content. Messages, text files, spreadsheets, presentations, reports, financials, etc, the list goes on. Usually organizations want to have a repository of all this content centralized somewhere (just in case a laptop breaks, gets lost or stolen for example). This leads to some kind of grouping and permission structure. You don’t want employees seeing each other’s HR records, unless they work for HR, same for Payroll, or unreleased quarterly numbers, etc. As this data grows it no longer becomes easy to simply navigate and a search engine is required to make sense of it all.

But what if your search engine returns 1000 results for a query and the user doing the search is supposed to only have access to see 4 things? How do you handle this? Check the user permissions on each file realtime? Slow. Pre-calculate all document permissions for a user on login? Slow and what if new documents are created or permissions change between logins? Does the system scale at 1M documents, 10M documents, 100M documents?
Continue reading

Tagged , , , , ,
Follow

Get every new post delivered to your Inbox.

Join 1,576 other followers