Visualizing a Network with Cypher and D3.js

We’ve seen some pretty nice visualizations of nodes and their immediate neighbors, but we want to be able to visualize more. So we’re going to prepare a 200 node network, use Cypher to extract the data we want and visualize it with D3.js.

I’m not going to type in the names of 200 people, so we’ll create a method to generate some random text.

def generate_text(length=8)
  chars = 'abcdefghjkmnpqrstuvwxyz'
  name = ''
  length.times { |i| name << chars[rand(chars.length)] }
  name
end

We are once again going to use our the Batch command. We are creating 200 nodes with random names, and randomly connecting them to 0-9 other nodes. This is going to send between 500 and 1000 commands in one batched transaction. This is as big as we want to go for one batch command, if you wanted to create a bigger graph, breaking it up into multiple batches of 1000 commands would be best.

def create_graph
  neo = Neography::Rest.new
  graph_exists = neo.get_node_properties(1)
  return if graph_exists && graph_exists['name']

  names = 200.times.collect{|x| generate_text}

  commands = names.map{ |n| [:create_node, {"name" => n}]}
  names.each_index do |x| 
    follows = names.size.times.map{|y| y}
    follows.delete_at(x)
    follows.sample(rand(10)).each do |f|
      commands << [:create_relationship, "follows", "{#{x}}", "{#{f}}"]    
    end
  end

  batch_result = neo.batch *commands
end

I want this visualization to be a little more dynamic than the last one, so when you take a look you’ll randomly see the network of one of the nodes. With Cypher we’ll parametrize the node and pass it a random number to serve as our node_id. If you’ve been following along you’ll notice this query looks pretty similar to when we used Cypher to get friends of friends. We are just taking it up a notch by going to friends of friends of friends and also getting the count of others.

def follower_matrix
  neo = Neography::Rest.new
  cypher_query =  " START me = node({node_id})"
  cypher_query << " MATCH (me)-[?:follows]->(friends)-[?:follows]->(fof)-[?:follows]->(fofof)-[?:follows]->others"
  cypher_query << " RETURN me.name, friends.name, fof.name, fofof.name, count(others)"
  cypher_query << " ORDER BY friends.name, fof.name, fofof.name, count(others) DESC"
  neo.execute_query(cypher_query, {:node_id => 1 + rand(200)})["data"]
end  

When we get our data it is going to be an array of arrays with the count of others at the end. You are basically looking at paths without the relationships.

[["Max", "Ben", "Rob", "James", 5],
 ["Max", "Ben", "Rob", "Peter", 2],
 ["Max", "Ben", "Musannif", "Pramod", 2]
]

Our visualization is looking for a JSON object that looks more like this:

{"name":"Max",
 "children":[{"name":"Ben",
               "children":[{"name":"Rob",
                            "children":[{"name":"James",
                                         "size":2},
                                        {"name":"Peter",
                                         "size":5}
                                       ]},
                           {"name":"Musannif",
                            "children":[{"name":"Pramod",
                                         "size":2}
                                       ]}
                           ]},
            ]
}

We will do this in two steps. The first is to create a variable called data and fill it with a nested hash of our array of arrays.

get '/followers' do
  data = follower_matrix.inject({}) {|h,i| t = h; i.each {|n| t[n] ||= {}; t = t[n]}; h}
  with_children(data).to_json
end

We could have created a nice Nested Hash class, or used one of the existing gems that deal with Trees, but Ruby is nice enough to let us do it on one line. Don’t let that one liner freak you out, it is an just an old Ruby trick.

We need a way to turn this nested hash into the final format with children arrays and such, so we’ll write that method as follows:

def with_children(node)
  if node[node.keys.first].keys.first.is_a?(Integer)
    { "name" => node.keys.first,
      "size" => 1 + node[node.keys.first].keys.first 
     }
  else
    { "name" => node.keys.first, 
      "children" => node[node.keys.first].collect { |c| 
        with_children(Hash[c[0], c[1]]) } 
    }
  end
end

We are going to use the Size variable to figure out how big to make our leaf nodes. We start with 1 and add the count of others, because if there are zero others, our leaf node would have a zero size and we don’t want that.

Our visualization is about 120 lines long, so I won’t dump it all here. If you want to see it check it out on github.

It follows the D3 force collapsible example, but adds a little color using the size attribute so it won’t look so bland:

var colorscale = d3.scale.category10();

function color(d) {
  return d._children ? "#3182bd" : d.children ? "#c6dbef" : colorscale(d.size);
}

See it live on Heroku at http://evening-mountain-5731.herokuapp.com/index.html and make sure to refresh the page to see different results.

Tagged , , , , , , ,

9 thoughts on “Visualizing a Network with Cypher and D3.js

  1. […] Visualizing a Network with Cypher and D3.js by Max De Marzi. […]

  2. […] We’ve seen some pretty nice visualizations of nodes and their immediate neighbors, but we want to be able to visualize more. So we’re going to prepare a 200 node network, use Cypher to extract the data we want and visualize it with D3.js.    Javascript Read the original post on DZone… […]

  3. […] few weeks ago I showed you how to visualize a graph using the chord flare visualization and how to visualize a network using a force directed graph visualization from […]

  4. […] A few weeks ago I showed you how to visualize a graph using the chord flare visualization and how to visualize a network using a force directed graph visualization from […]

  5. MJ says:

    Do you know how to add labels to this visualization? I think it looks good and I’m using it in a program I’m writing but I need for labels to be shown for all the nodes.

  6. MJ says:

    By labels, I mean have them appear on the graph by default instead of when the user hovers over individual nodes with the mouse. The end product I’m looking for is being able to take a screenshot of the visualization and to place in a report or presentation.

  7. VK says:

    How do I use it for say huge number of nodes (in millions) ? I want to create a graph for huge number of nodes but putting them all in memory while traversing is not a good idea.
    Will the current approach scale for huge number of nodes ?

  8. […] Visualizing a network with Cypher and d3.js […]

  9. Ke Yang says:

    I got the following problem .How to resolve it?
    root@ubuntu:/home/yangke/Program/D3/d3_js_network-master# bundle install
    Fetching source index for http://rubygems.org/
    Could not find twitter-2.1.0 in any of the sources
    root@ubuntu:/home/yangke/Program/D3/d3_js_network-master# rackup
    /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require': no such file to load — neography (LoadError)
    from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in `require’
    from ./network.rb:2
    from /home/yangke/Program/D3/d3_js_network-master/config.ru:2:in `require’
    from /home/yangke/Program/D3/d3_js_network-master/config.ru:2
    from /usr/lib/ruby/vendor_ruby/rack/builder.rb:51:in `instance_eval’
    from /usr/lib/ruby/vendor_ruby/rack/builder.rb:51:in `initialize’
    from /home/yangke/Program/D3/d3_js_network-master/config.ru:1:in `new’
    from /home/yangke/Program/D3/d3_js_network-master/config.ru:1

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 1,682 other followers

%d bloggers like this: