Graph Visualization and Neo4j – Part Three

Like I promised in my previous post, I wanted to do a little something on D3.js.

We are going to take one of their example visualizations and visualize a follows graph.

To create our graph, we will take the names of 20 people: create nodes for them, add them to an index, and randomly link them together.

Notice we are using the Neography Batch command to create the whole graph at once.

def create_graph
  neo = Neography::Rest.new
  graph_exists = neo.get_node_properties(1)
  return if graph_exists && graph_exists['name']

  names = %w[Max Agam Lester Musannif Adel Andrey Ryan James Bruce Tim Pinaki Mark Peter Anne Helene Corey Ben Rob Pramod Prasanna]

  commands = names.map{ |n| [:create_node, {"name" => n}]}
  names.each_index do |x| 
    commands << [:add_node_to_index, "nodes_index", "type", "User", "{#{x}}"]
    follows = names.size.times.map{|y| y}
    follows.delete_at(x)
    follows.sample(1 + rand(5)).each do |f|
      commands << [:create_relationship, "follows", "{#{x}}", "{#{f}}"]    
    end
  end

  batch_result = neo.batch *commands
end

We won’t be making the mistake of leaving the create_graph method publicly accessible again, so we’ll create a Rake task for it.

require 'neography/tasks'
require './d3.rb'

namespace :neo4j do
  task :create do
    create_graph
  end
end

We will use Cypher to create a follower matrix, which we will use to populate our D3 script.

def follower_matrix
  neo = Neography::Rest.new
  cypher_query =  " START a = node:nodes_index(type='User')"
  cypher_query << " MATCH a-[:follows]->b"
  cypher_query << " RETURN a.name, collect(b.name)"
  neo.execute_query(cypher_query)["data"]
end  

The collect function returns a string with an array inside it, so we have to some string wrangling to turn it into a proper array and then convert everything to JSON.

get '/follows' do
  follower_matrix.map{|fm| {"name" => fm[0], "follows" => fm[1][1..(fm[1].size - 2)].split(", ")} }.to_json
end

Our D3 function is a small variation on the chord flare example in the D3 github repository:

var r1 = 960 / 2,
    r0 = r1 - 120;

var fill = d3.scale.category20c();

var chord = d3.layout.chord()
    .padding(.04)
    .sortSubgroups(d3.descending)
    .sortChords(d3.descending);

var arc = d3.svg.arc()
    .innerRadius(r0)
    .outerRadius(r0 + 20);

var svg = d3.select("body").append("svg")
    .attr("width", r1 * 2)
    .attr("height", r1 * 2)
  .append("g")
    .attr("transform", "translate(" + r1 + "," + r1 + ")");

function fade(opacity) {
  return function(g, i) {
    svg.selectAll("g path.chord")
        .filter(function(d) {
          return d.source.index != i && d.target.index != i;
        })
      .transition()
        .style("opacity", opacity);
  };
}

function draw(follows) {
  var indexByName = {},
      nameByIndex = {},
      matrix = [],
      n = 0;

  function name(name) {
    return name
  }

  // Compute a unique index for each name.
  follows.forEach(function(d) {
    d = name(d.name);
    if (!(d in indexByName)) {
      nameByIndex[n] = d;
      indexByName[d] = n++;
    }
  });

  // Construct a square matrix counting relationships.
  follows.forEach(function(d) {
    var source = indexByName[name(d.name)],
        row = matrix1;
    if (!row) {
     row = matrix1 = [];
     for (var i = -1; ++i < n;) row[i] = 0;
    }
    d.follows.forEach(function(d) { row[indexByName[name(d)]]++; });
  });

  chord.matrix(matrix);

  var g = svg.selectAll("g.group")
      .data(chord.groups)
    .enter().append("g")
      .attr("class", "group");

  g.append("path")
      .style("fill", function(d) { return fill(d.index); })
      .style("stroke", function(d) { return fill(d.index); })
      .attr("d", arc);

  g.append("text")
      .each(function(d) { d.angle = (d.startAngle + d.endAngle) / 2; })
      .attr("dy", ".35em")
      .attr("text-anchor", function(d) { return d.angle > Math.PI ? "end" : null; })
      .attr("transform", function(d) {
        return "rotate(" + (d.angle * 180 / Math.PI - 90) + ")"
            + "translate(" + (r0 + 26) + ")"
            + (d.angle > Math.PI ? "rotate(180)" : "");
      })
      .text(function(d) { return nameByIndex[d.index]; });

  svg.selectAll("path.chord")
      .data(chord.chords)
    .enter().append("path")
      .attr("class", "chord")
      .style("stroke", function(d) { return d3.rgb(fill(d.source.index)).darker(); })
      .style("fill", function(d) { return fill(d.source.index); })
      .attr("d", d3.svg.chord().radius(r0));

}

d3.json("follows",draw);

All of the code is available on Github.
Finally, we’ll put all this on Heroku, like I’ve shown you before:

git clone git@github.com:maxdemarzi/d3_js_intro.git
cd d3_js_intro
bundle install
heroku create --stack cedar
heroku addons:add neo4j
git push heroku master
heroku run rake neo4j:create

Pretty isn’t it?

Update: The visualization was hard to follow with all those paths, so I added a mouse over function to fade the paths I am not interested in.
Click the image to see the live version and put your mouse over one of the users to see:

Tagged , , , , ,

20 thoughts on “Graph Visualization and Neo4j – Part Three

  1. That graphic is gorgeous! keep it up man, nice work.

  2. geronimo1271 says:

    very cool!!!

  3. Great series!

    You mention: “We won’t be making the mistake of leaving the create_graph method publicly accessible again, so we’ll create a Rake task for it.”

    Is this a coding or security issue? Just curious.

    Thanks for all the excellent work you are sharing!

    Patrick

  4. maxdemarzi says:

    A bit of both. For this exercise, we are creating a random static graph and we only need to do this once, so it might as well be a rake task. I had left the create_graph method accessible by going to the site and the url mysite.com/create_graph in a previous example, not expecting people to go there just to try it out.

    This example could be more fun if we used the actual Twitter data from people who visited. It would be a good http://neo4j-challenge.herokuapp.com . Anybody up to it?

  5. Lyndon Adams says:

    This is just great!! Keep up the good work.

  6. […] few weeks ago I showed you how to visualize a graph using the chord flare visualization and how to visualize a network using a force directed graph […]

  7. […] De Marzi writes: A few weeks ago I showed you how to visualize a graph using the chord flare visualization and how to visualize a network using a force directed graph […]

  8. Mahdi says:

    is there any way to create this project with java ?

    • maxdemarzi says:

      Absolutely. The cypher query can be called from any language, and the rest is just boiler plate stuff to create a random graph, put it in a JSON object and pass it on to the Javascript visualization.

  9. Mickey says:

    Hi Max,
    Where can I find an example how to create the graph on the head of the post (grey one) using Neo and D3?
    Thanks

  10. […] are re-using the D3 Chord visualization we saw before and that’s all there is too […]

  11. Nilanjan says:

    It seems like the chord visualization has a limit to the number of nodes/edges. Beyond a point the browser will start to groan?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 1,682 other followers

%d bloggers like this: