Max De Marzi

Je n’ai fait celle-ci plus longue que parce que je n’ai pas eu le loisir de la faire plus courte. I would have written a shorter letter, but I did not have the time. Written by Blaise Pascal is often misattributed to Mark Twain. It reminds us to try to be brief. Too many people never learn this.

I think the point would be driven home by another quote from Pascal:

When I consider the short duration of my life, swallowed up in the eternity before and after, the small space which I fill, or even can see, engulfed in the infinite immensity of spaces whereof I know nothing, and which know nothing of me, I am terrified…

Memoria hospitis unius diei prætereuntis

Ok, maybe it’s time I got to the point… These methods names are way too long:

NodeGetNeighborsForDirectionForType
NodeGetNeighborsByIdForDirectionForType
NodeGetLinksByIdForDirectionForTypes
NodeGetRelationshipsForDirectionForType
NodeGetRelationshipsByIdForDirectionForTypes
LinksGetLinksForDirectionForType

Having to worry about “ById” and “ForDirection”, “ForType”, “ForTypes”, etc is now a thing of the past. Let me explain. No, there is too much. Let me sum up. Lua doesn’t let us overload functions out of the gate, but you can enable it with about 40 lines of code. Sol let’s us overload using sol::overload but the documentation doesn’t mention how to overload non-static member functions. We are calling everything from a Shard so non-static member functions is pretty much all we have. For example to add the ability to get the Property of a Relationship I would add a function to our Lua environment passing the name, the C++ function, and “this” as the first argument.

lua.set_function("RelationshipGetProperty", &Shard::RelationshipGetPropertyViaLua, this);

Problem is sol::overload can’t take “this”. I searched around and found this github issue that mentioned lambdas…

this->lua.set_function("nonStaticfunction", [this]() {this->nonStaticFunction();});

Ok, great I thought. I just have to wrap everything in a lambda and do this then:

        lua.set_function("overloaded",
             sol::overload(
               [this](uint64_t id) {this->NodeGetLinksByIdViaLua(id);}
               ,[this](std::string type, std::string key) {this->NodeGetLinksViaLua(type, key);}));

it compiled but didn’t work for me so I asked…and thankfully Rochet2 figured it out for me.

lua.set_function("overloaded", sol::overload(
    [this](uint64_t id) { return this->NodeGetLinksByIdViaLua(id); },
    [this](std::string type, std::string key) { return this->NodeGetLinksViaLua(type, key); }
));

I was missing the “return“. I was returning nothing. Of course it wasn’t “working“. This is what happens when you code on your nights and weekends. Anyway. Now if you want to get the Relationships of a Node, you just say “NodeGetRelationships” and pass in whatever you want. An id, a type and key, plus a direction or a type or a list of types or whatever combination floats your boat. It all goes to NodeGetRelationships and we’ll take it from there.

        lua.set_function("NodeGetRelationships", sol::overload(
           [this](uint64_t id) { return this->NodeGetRelationshipsByIdViaLua(id); },
           [this](uint64_t id, Direction direction) { return this->NodeGetRelationshipsByIdForDirectionViaLua(id, direction); },
           [this](uint64_t id, Direction direction, std::string rel_type) { return this->NodeGetRelationshipsByIdForDirectionForTypeViaLua(id, direction, rel_type); },
           [this](uint64_t id, Direction direction, std::vector<std::string> rel_types) { return this->NodeGetRelationshipsByIdForDirectionForTypesViaLua(id, direction, rel_types); },
           [this](uint64_t id, std::string rel_type) { return this->NodeGetRelationshipsByIdForTypeViaLua(id, rel_type); },
           [this](uint64_t id, std::vector<std::string> rel_types) { return this->NodeGetRelationshipsByIdForTypesViaLua(id, rel_types); },
           [this](std::string type, std::string key) { return this->NodeGetRelationshipsViaLua(type, key); },
           [this](std::string type, std::string key, Direction direction) { return this->NodeGetRelationshipsForDirectionViaLua(type, key, direction); },
           [this](std::string type, std::string key, Direction direction, std::string rel_type) { return this->NodeGetRelationshipsForDirectionForTypeViaLua(type, key, direction, rel_type); },
           [this](std::string type, std::string key, Direction direction, std::vector<std::string> rel_types) { return this->NodeGetRelationshipsForDirectionForTypesViaLua(type, key, direction, rel_types); },
           [this](std::string type, std::string key, std::string rel_type) { return this->NodeGetRelationshipsForTypeViaLua(type, key, rel_type); },
           [this](std::string type, std::string key, std::vector<std::string> rel_types) { return this->NodeGetRelationshipsForTypesViaLua(type, key, rel_types); }
           ));

Going back to the long function names, check out the before and after. We go from those 40+ character monsters to only about 20 characters. Maybe I should steal a page from Gremlin and get down to “VgV”, “VgL”, “VgE”, “LgL” or maybe not:

Take a look at those LDBC Social Network Benchmark Short Queries now… a little bit nicer eh? According to the Sol documentation we lose a tiny bit of performance when overloading as it has to figure out which method we want, but it must be measured in microseconds because I didn’t notice anything worth worrying about.

You may have noticed a little “clock” on the Rage DB UI that lets you know how long it took to run the query. This functionality is purely browser side. I am using the Performance Timeline specification to capture the duration of requests sent to the “lua” endpoint.

It is supported by all major browsers except Internet Explorer (seriously who still uses that?). The way it works is that we create a Performance Observer object that watches for these “fetch” requests going to “lua” and updates the timer with the duration.

const observer = new PerformanceObserver((list) => {
    for (const entry of list.getEntries()) {
        if (entry.initiatorType === "fetch" && entry.name.endsWith("lua")) {
            document.getElementById('timer').innerHTML = timeUnits(entry.duration);
        }
    }
});

Then we tell it to observe:

observer.observe({
    entryTypes: ["resource"]
});

If you click on any of the requests you can get more fine grained details from the browser. We’re just capturing the final duration rounded off. Notice in this case our query started returning an answer in under 3 ms, but the overall duration was 7 ms due to queuing.

I think the clock is a great visual aid for “rough” estimates and comparisons of how long a query takes to run. But if you are trying to decide which query is faster and they are 1 to 3 ms different from each other… maybe you should use wrk or maybe you should scroll back up to the top of this blog post, read that second Blaise Pascal quote and just flip a coin.