Max De Marzi

Last year eBay built a chatbot using Neo4j. Unfortunately we have grown so big I didn’t get a chance to work on that project and kinda feel left out. So I decided I’m going to build my own chatbot with Neo4j. As usual I’ve never done this before, have very little idea what I’m doing, have no team, and have barely any time to get this done. So with those disclaimers out of the way, let’s see what we can do.

We’ll build our own shopping chat bot but to make things a little bit simpler we aren’t going to use the massive catalog of eBay or any of the retailers. Instead we’ll use a much smaller catalog. The combined catalog of the Shadowrun role playing game as collected and curated by Chummer.

We’ll build a stored procedure to handle most of the logic, and a website to demo the work. The first thing we are going to need is to try to understand what the user is telling us. For that we need some Natural Language Processing framework. Luckily we have OpenNLP available and folks have written some handy blog posts on how to use it to build chatbots.

For those of you adventurous enough, follow along with the source code. What the user is trying to tell us is called an “intent”. Our chat bot should be able to recognize and handle a set of intents. We’ll start with the simplest intents and work our way up from there. Besides the intent of the user we need to recognize some of the specific things they may be saying. They could be talking about a product or category or size, etc. We need to use named entity recognition (NER) to find these in the text.

So we need to get and train some models to do all this work. Let’s build a procedure called train to do just that. It will take two parameters, one is the location of a directory where we can find some pre-trained models, and the second is the intents directory so we can build models for these intents.

 
    @Procedure(name = "com.maxdemarzi.train", mode = Mode.READ)
    @Description("CALL com.maxdemarzi.train(model_directory, intents_directory)")
    public Stream<StringResult> train(@Name(value = "model_directory", defaultValue = "") String modelDirectory,
                                      @Name(value = "intents_directory", defaultValue = "") String intentsDirectory) {

We need to start with the basics. We are going to get a bunch of text from the user, we need to split it up into sentences. For that we need a “sentencizer”. We could build one ourselves, but instead we’re going to take advantage of some pre-built models to make our lives easier. We’ll download the “en-sent.bin”, put that in the models directory and initialize it:

 
            modelIn =  new FileInputStream(sent);
            SentenceModel sentenceModel = new SentenceModel(modelIn);
            sentencizer = new SentenceDetectorME(sentenceModel);

We’re going to need a tokenizer to split up each sentence into smaller parts. Then we’ll need to recognize parts of speech, and we’ll need a lemmatizer to simplify the words into their base. Thankfully we can download models for all of these, and they are stored in the resources folder of our repository.

 
            modelIn = new FileInputStream(token);
            TokenizerModel model = new TokenizerModel(modelIn);
            tokenizer = new TokenizerME(model);

            modelIn =  new FileInputStream(maxent);
            POSModel posModel = new POSModel(modelIn);
            partOfSpeecher = new POSTaggerME(posModel);

            modelIn =  new FileInputStream(lemma);
            LemmatizerModel lemmaModel = new LemmatizerModel(modelIn);
            lemmatizer = new LemmatizerME(lemmaModel);

Now we need to build files to understand intents. We’ll start with the “greeting” intent which looks like this:

 
good morning
g’day mate
hello
hey
hi
...

Basically it’s a collection of lines where each line could be construed as a greeting. The longer the list, the more greetings we are able to recognize. We would do the same for completing a conversation:

 
adieu
adios
all right then
back later
bye
catch you later
cya
good bye
...

Each intent would go into a separate file, and we would gather these into a stream of DocumentSamples. From here we would use these samples to train a Document Categorizer:

 
ObjectStream<DocumentSample> combinedDocumentSampleStream = ObjectStreamUtils.concatenateObjectStream(categoryStreams);
DoccatFactory factory = new DoccatFactory(new FeatureGenerator[] { new BagOfWordsFeatureGenerator() });

DoccatModel doccatModel = DocumentCategorizerME.train("en", combinedDocumentSampleStream, trainingParams, factory);
combinedDocumentSampleStream.close();
DocumentCategorizerME categorizer = new DocumentCategorizerME(doccatModel);

We saw some intents that are pretty straight forward, but some intents would need to refer to something in particular. For example for the intent called “category inquiry” we need to know what product category they are referring to. We can mark up the file so it looks like this:

 
show me your <START:category> shotguns <END>
show me the <START:category> rifles <END>
what kind of <START:category> rides <END> you got
what <START:category> heavy pistols <END> do you have
...

For each type of object we can recognize, we need to build and train a TokenNameFinderModel and a NameFinderME. So we need a list of these:

 
List<NameFinderME> nameFinderMEs

Not all intents can have all objects appear in them, so we separate these out. For example, “Products” can appear in the price inquiry intent and the product inquiry intent:

 
HashMap<String,ArrayList<String>> slots = new HashMap<>();
slots.put("product", new ArrayList<String>() {{
   add("price_inquiry");
   add("product_inquiry");
}});

For each one of these, we’ll read all the intents that can contain them, and add them to our list:

 
ObjectStream<String> lineStream = new PlainTextByLineStream(new MarkableFileInputStreamFactory(trainingFile), "UTF-8");
ObjectStream<NameSample> nameSampleStream = new NameSampleDataStream(lineStream);
...
ObjectStream<NameSample> combinedNameSampleStream = ObjectStreamUtils.concatenateObjectStream(nameStreams);

TokenNameFinderModel tokenNameFinderModel = NameFinderME.train("en", slot.getKey(), combinedNameSampleStream, trainingParams, new TokenNameFinderFactory());
combinedNameSampleStream.close();
nameFinderMEs.add(new NameFinderME(tokenNameFinderModel));

We can also get a few free models for dates, money and people in case we need them later. Finally we will end our stored procedure and return “Training Complete!”. In the current dataset I have, this takes about 5 seconds. It make take longer once we’re done and have added more data.

 
modelIn = new FileInputStream(date);
TokenNameFinderModel dateModel = new TokenNameFinderModel(modelIn);
nameFinderMEs.add(new NameFinderME(dateModel));
...            
return Stream.of(new StringResult("Training Complete!"));

Ok, so far so good. We have trained a bunch of models. Now we need to test if they correctly guess the intent of the user and recognize entities. We’ll create another stored procedure that takes a string and test it out.

 
    @Procedure(name = "com.maxdemarzi.intents", mode = Mode.READ)
    @Description("CALL com.maxdemarzi.intents(String text)")
    public Stream<IntentResult> intents(@Name(value = "text") String text) {
        ArrayList<IntentResult> results = new ArrayList<>();
        findIntents(text, results);
        return results.stream();
    }

The findIntents method does all the work. First it finds the sentences in the text, then builds the part of speech tags, then lemmatizes each word to it’s base and gets the most probable category from the potential outcomes:

 
    private void findIntents(String text, ArrayList<IntentResult> results) {
        String[] sentences = sentencizer.sentDetect(text);

        for (String sentence : sentences) {
            // Separate words from each sentence using tokenizer.
            String[] tokens = tokenizer.tokenize(sentence);

            // Tag separated words with POS tags to understand their grammatical structure.
            String[] posTags = partOfSpeecher.tag(tokens);

            // Lemmatize each word so that it is easy to categorize.
            String[] lemmas = lemmatizer.lemmatize(tokens, posTags);

            double[] probabilitiesOfOutcomes = categorizer.categorize(lemmas);
            String category = categorizer.getBestCategory(probabilitiesOfOutcomes);

Now that we have the best category, we also need to figure out if it finds any entities. For each nameFinder we created earlier we check the tokens and see if any of them match. Then we put them together in an IntentResult and add it to the list.

 
            List<Map<String, Object>> args = new ArrayList<>();

            for (NameFinderME nameFinderME : nameFinderMEs) {
                Span[] spans = nameFinderME.find(tokens);
                String[] names = Span.spansToStrings(spans, tokens);
                for (int i = 0; i < spans.length; i++) {
                    HashMap<String, Object> arg = new HashMap<>();
                    arg.put(spans[i].getType(), names[i]);
                    args.add(arg);
                }
            }

            results.add(new IntentResult(category, args));

Let’s write a test for this. Starting with a simple greeting:

 
            // Given I've started Neo4j and trained the models
            Session session = driver.session();
            session.run( "CALL com.maxdemarzi.train" );
            
            // When I use the procedure
            StatementResult result = session.run( "CALL com.maxdemarzi.intents($text)",
                    parameters( "text", "Hello?" ) );

            // Then I should get what I expect
            assertThat(result.single().get("intent").asString()).isEqualTo("greeting");

Cool, that passes, now how about one with an entity:

 
            result = session.run( "CALL com.maxdemarzi.intents($text)",
                    parameters( "text", "show me your shotguns" ) );
            Record record = result.single();
            assertThat(record.get("intent").asString()).isEqualTo("category_inquiry");
            List<Object> args = record.get("args").asList();
            Map<String, Object> arg = (Map<String, Object>)args.get(0);
            assertThat(arg.containsKey("category"));
            assertThat(arg.get("category").toString()).isEqualTo("shotguns");

That passes as well. Pretty neat. So far we’ve built a way to take text and turn it into an intent and named entities, so we can “hear” what the user is trying to tell us. Now how do we respond? Ah well this is where the Graph comes in. I’ll go over my approach in Part 2, so stay tuned. If you can’t wait, the source code is a bit ahead of the blog posts so go take a look.