Max De Marzi

Today I want to introduce you to VivaGraphJS – a JavaScript Graph Drawing Library made by Andrei Kashcha of Yasiv. It supports rendering graphs using WebGL, SVG or CSS formats and currently supports a force directed layout. The Library provides an API which tracks graph changes and reflect changes on the rendering surface which makes it fantastic for graph exploration.

Today we will be integrating it with Neo4j and the Alchemy API.

The Alchemy API provides a set of natural language processing tools, and we’ll make use of their Entity Extraction capabilities.

We are going to take a look at the news of the world, extract the entities mentioned in them, connect them all together in Neo4j and visualize them. To get the news, we’ll use Feedzilla.

A sidekiq job will run every six hours to collect the latest 100 news articles:

module Job
  class GetNews
    include Sidekiq::Worker
    sidekiq_options :retry => false

    def perform
      Job::GetNews.perform_in(6.hours)

      feed = HTTPClient.get("http://api.feedzilla.com/v1/categories/19/articles.json?order=date&count=100&title_only=1")
      parsed_feed = Oj.load(feed.body)
      parsed_feed["articles"].each do |article|
        Job::GetArticle.perform_async(article["url"])
      end
    end

  end
end

From each article, we’ll get the story and pass it on to Alchemy API and ask it to find the entities in the article.

@entities = Oj.load(HTTPClient.post_content("http://access.alchemyapi.com/calls/url/URLGetRankedNamedEntities",
      {:url => article_url, 
       :apikey => ENV['ALCHEMY_API'],
       : outputMode => "json"}),
       {:'Accept-encoding' => "gzip"})["entities"]

We’ll create a node for each article and connect each entity found by a “MENTIONED” relationship:

commands = []
@batch_result.each do |b|
  commands << [:create_relationship, "MENTIONED", @article_node, b["body"]["self"].split("/").last]
end

Some of these entities have interesting descriptions, so we’ll query dbpedia for more information.

Moving on to our front-end application, we’ll provide an auto-complete search box which will query the Entities index of our graph:

  get '/search' do 
    content_type :json
    neo = Neography::Rest.new    

    cypher = "START me=node:entities({query}) 
              RETURN ID(me), me.text
              ORDER BY me.text
              LIMIT 15"

    neo.execute_query(cypher, {:query => "text:*#{params[:term]}* OR uri:*#{params[:term]}*" })["data"].map{|x| { label: x[1], value: x[0]}}.to_json   
  end

We’ll also provide an end-point to pull a node and all the nodes connected to it and return a JSON object with this data:

get '/edges/:id' do
    content_type :json
    neo = Neography::Rest.new    

    cypher = "START me=node(#{params[:id]}) 
              MATCH me -- related
              RETURN ID(me), me.text, me.description, me.type, ID(related), related.text, related.description, related.type"

    connections = neo.execute_query(cypher)["data"]   
    connections.collect{|n| {"source" => n[0], "source_data" => {:label => n[1], 
                                                                 :description => n[2],
                                                                 :type => n[3] },
                             "target" => n[4], "target_data" => {:label => n[5], 
                                                                 :description => n[6],
                                                                 :type => n[7]}} }.to_json
  end

Our javascript will contain the calls to Vivagraph:

var graph = Viva.Graph.graph();	

  var layout = Viva.Graph.Layout.forceDirected(graph, {
      springLength:100,
      springCoeff:0.0001,
      dragCoeff:0.02,
      gravity:-1
  });	

Our nodes will display both the Text found in the entity as well as an image to represent the entity type.

var ui = Viva.Graph.svg('g'),
        svgText = Viva.Graph.svg('text').attr('y', '-4px').text(node.data.label),
        img = Viva.Graph.svg('image')
           .attr('width', 32)
           .attr('height', 32)
           .link('/img/' + node.data.type + '.png');

    ui.append(svgText);
    ui.append(img);

We’ll add a mouseover event to update the side-panel with the description of our node:

$(ui).hover(function() { // mouse over
	                    highlightRelatedNodes(node.id, true);
	                    $('#explanation').html(node.data.description);
	                }, function() { // mouse out
	                    highlightRelatedNodes(node.id, false);
	                });

…and to load the nodes related to this entity on click:

$(ui).click(function() { 
				        console.log("click", node);
			        	if (!node || !node.position) return;
			        	renderer.rerender();
			        	loadData(graph,node.id);
			}
	);

As usual, the code is available on github, and you can see it running live on Heroku.

Check out some more of the Vivagraph.js demos:

• Amazon Visualization– Shows related products on Amazon.com, uses SVG as graph output
• YouTube Visualization – Shows related videos from YouTube. SVG based.
• Facebook Visualization – friendship visualization on Facebook. WebGL based.
• Graph Viewer – visualization of sparse matrices collection of the University of Florida. WebGL based.
• Vkontakte Visualization – friendship visualization of the largest social network in Russia vk.com. WebGL based.