Using Sigma.js with Neo4j

I’ve done a few posts recently using D3.js and now I want to show you how to use two other great Javascript libraries to visualize your graphs. We’ll start with Sigma.js and soon I’ll do another post with Three.js.

We’re going to create our graph and group our nodes into five clusters. You’ll notice later on that we’re going to give our clustered nodes colors using rgb values so we’ll be able to see them move around until they find their right place in our layout. We’ll be using two Sigma.js plugins, the GEFX (Graph Exchange XML Format) parser and the ForceAtlas2 layout.

You can see what a GEFX file looks like below. Notice it comes from Gephi which is an interactive visualization and exploration platform, which runs on all major operating systems, is open source, and is free.

<?xml version="1.0" encoding="UTF-8"?>
<gexf xmlns="http://www.gephi.org/gexf" xmlns:viz="http://www.gephi.org/gexf/viz">
  <graph defaultedgetype="directed" idtype="string" type="static">
    <nodes count="500">
      <node id="1" label="tnabcuff">
        <viz:size value="12.0"/>
        <viz:color b="113" g="42" r="78"/>
        <viz:position x="-195" y="-53"/>
      </node>
      <node id="2" label="khnvxggh">
        <viz:size value="14.0"/>
        <viz:color b="237" g="250" r="36"/>
        <viz:position x="277" y="-73"/>
      </node>
      ...
    </nodes>
    <edges count="2985">
      <edge id="0" source="1" target="11"/>
      <edge id="1" source="1" target="21"/>
      <edge id="2" source="1" target="31"/>
      ...
    </edges>
  </graph>
</gexf>

In order to build this file, we will need to get the nodes and edges from the graph and create an XML file.

get '/graph.xml' do
  @nodes = nodes  
  @edges = edges
  builder :graph
end

We’ll use Cypher to get our nodes and edges:

def nodes
  neo = Neography::Rest.new
  cypher_query =  " START node = node:nodes_index(type='User')"
  cypher_query << " RETURN ID(node), node"
  neo.execute_query(cypher_query)["data"].collect{|n| {"id" => n[0]}.merge(n[1]["data"])}
end  

We need the node and relationship ids, so notice I’m using the ID() function in both cases.

def edges
  neo = Neography::Rest.new
  cypher_query =  " START source = node:nodes_index(type='User')"
  cypher_query << " MATCH source -[rel]-> target"
  cypher_query << " RETURN ID(rel), ID(source), ID(target)"
  neo.execute_query(cypher_query)["data"].collect{|n| {"id" => n[0], 
                                                       "source" => n[1], 
                                                       "target" => n[2]} 
                                                      }
end

So far we have seen graphs represented as JSON, and we’ve built these manually. Today we’ll take advantage of the Builder Ruby Gem to build our graph in XML.

xml.instruct! :xml
xml.gexf 'xmlns' => "http://www.gephi.org/gexf", 'xmlns:viz' => "http://www.gephi.org/gexf/viz"  do
  xml.graph 'defaultedgetype' => "directed", 'idtype' => "string", 'type' => "static" do
    xml.nodes :count => @nodes.size do
      @nodes.each do |n|
        xml.node :id => n["id"],    :label => n["name"] do
	   xml.tag!("viz:size",     :value => n["size"])
	   xml.tag!("viz:color",    :b => n["b"], :g => n["g"], :r => n["r"])
	   xml.tag!("viz:position", :x => n["x"], :y => n["y"]) 
	 end
      end
    end
    xml.edges :count => @edges.size do
      @edges.each do |e|
        xml.edge:id => e["id"], :source => e["source"], :target => e["target"] 
      end
    end
  end
end

You can get the code on github as usual and see it running live on Heroku at neosigma.herokuapp.com. You will want to see it live on Heroku so you can see the nodes in random positions and then move to form clusters. Use your mouse wheel to zoom in, and click and drag to move around.

Credit goes out to Alexis Jacomy and Mathieu Jacomy.

You’ve seen me create numerous random graphs, but for completeness here is the code for this graph. Notice how I create 5 clusters and for each node I assign half its relationships to other nodes in their cluster and half to random nodes? This is so the ForceAtlas2 layout plugin clusters our nodes neatly.

def create_graph
  neo = Neography::Rest.new
  graph_exists = neo.get_node_properties(1)
  return if graph_exists && graph_exists['name']

  names = 500.times.collect{|x| generate_text}
  clusters = 5.times.collect{|x| {:r => rand(256),
                                  :g => rand(256),
                                  :b => rand(256)} }
  commands = []
  names.each_index do |n|
    cluster = clusters[n % clusters.size]
    commands << [:create_node, {:name => names[n], 
                                :size => 5.0 + rand(20.0), 
                                :r => cluster[:r],
                                :g => cluster[:g],
                                :b => cluster[:b],
                                :x => rand(600) - 300,
                                :y => rand(150) - 150
                                 }]
  end
 
  names.each_index do |from| 
    commands << [:add_node_to_index, "nodes_index", "type", "User", "{#{from}}"]
    connected = []

    # create clustered relationships
    members = 20.times.collect{|x| x * 10 + (from % clusters.size)}
    members.delete(from)
    rels = 3
    rels.times do |x|
      to = members[x]
      connected << to
      commands << [:create_relationship, "follows", "{#{from}}", "{#{to}}"]  unless to == from
    end    

    # create random relationships
    rels = 3
    rels.times do |x|
      to = rand(names.size)
      commands << [:create_relationship, "follows", "{#{from}}", "{#{to}}"] unless (to == from) || connected.include?(to)
    end

  end

  batch_result = neo.batch *commands
end
Tagged , , , , , ,

10 thoughts on “Using Sigma.js with Neo4j

  1. [...] Using Sigma.js with Neo4j by Max De Marzi. [...]

  2. [...] followed Max De Marxi’s blog post to create a GEFX (Graph Exchange XML Format) file to use in gephi although I later learned that you [...]

  3. [...] initially tried to visualise the data in sigma.js but that didn’t work that well here – I think it’s much better when we actually [...]

  4. Semiografo says:

    I didn’t understand well what is a cluster and how does it work. In pure graph theory, usually all we have are nodes and edges (arrows in a directed graph). So, in short, where the cluster fits in the theory?

  5. AmP says:

    in Git, you have a file ‘neo_sigma.js’ under public folder.

    Few Question:
    1. What this line actually do: sigInst.parseGexf(‘graph.xml’);
    2. Where is this file ‘graph.xml’ comes from. Is it generated on runtime.
    3. Is it mandatory to use gexf file format for node and edges to display graph using sigma.js
    4.Can we any other way to plot the graph without using gexf format

    Thanks
    Amp

    • maxdemarzi says:

      1. Load and parses the data.
      2. on runtime using the builder gem
      3. No, you can use a JSON format for input.
      4. See https://github.com/jacomyal/sigma.js/tree/master/plugins you can get a dummy gexf file convert it to the json format using the python script there and then you’ll have a good idea of what the format looks like.

      • AmP says:

        Thanks max for the reply and I gave a shot to json format by using python script.

        Let me elaborate more on my point (4) w.r.t above line.
        Ref: les_miserables.gexf

        #GEXF Snippet:

        #JSON Snippet(using python)
        {
        “@attributes”: {
        “id”: “1.0”,
        “label”: “Napoleon”
        },
        “attvalues”: {
        “attvalue”: [
        {
        "@attributes": {
        "for": "authority",
        "value": "0.0034188034"
        }
        },
        {
        "@attributes": {
        "for": "hub",
        "value": "0.0034188034"
        }
        }
        ]}
        },

        If you see the JSONoutput, there are NO x,y and z coordinates when compared with GEXF.
        My question is:
        1. How does sigma.js will plot the graph (nodes & edges) when there are no coordinates in converted JSON?
        2. Also, can we draw a graph using sigma on a pre-define coordinates for a dynamic data, where parent-child relation needs to maintain. See Example: http://demos.michelepasin.org/nature/d3tree

        Thanks.

  6. pducrot says:

    In you demo you first a GEFX file containing the graph, then parse it for display with sigma.js. Would it be possible to directly get the JSON formatted graph from neo4j using cypher? How would you do that?

  7. pducrot says:

    Hi, thanks for this piece of code, looks very interesting. You first generate a GEFX file containing the graph then parse it before display in sigma.js. Would it be possible to directly obtain the graph from neo4j via the REST API and your cypher query in a JSON format and then display the graph with sigma.js without using any temporary file?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 1,579 other followers

%d bloggers like this: