Typically we want to Reduce, Reuse and Recycle to help the environment. But today we are going to Reduce, Reuse and Recycle the Lua Sandbox Environment to give us two additional sets of permissions. The first is “Read Write” in which a user can read and write to the database but cannot create new types of nodes or relationships or data types. The second is “Read Only” which does what it sounds like.
While we’re here, we’re going to one graph, two graph, rage graph, blue graph our way to multi database support. Let’s jump in:
Let’s start of by creating an enum to hold our Permission options:
enum Permission {
ADMIN, WRITE, READ
};
Next we will go to our Sandbox.h file and split the Graph functions into 3 lists. One to contain functions that modify the schema of the graph:
inline static const std::vector<std::string> ALLOWED_GRAPH_ADMIN_FUNCTIONS = {
"NodePropertyTypeAdd",
"NodePropertyTypeDelete",
"NodeTypeInsert",
"RelationshipPropertyTypeAdd",
"RelationshipPropertyTypeDelete",
"RelationshipTypeInsert",
};
Another that contains functions that allow writes to the graph:
inline static const std::vector<std::string> ALLOWED_GRAPH_WRITE_FUNCTIONS = {
"NodeAdd",
"NodeDeleteProperties",
"NodeDeleteProperty",
"NodeRemove",
"NodeResetPropertiesFromJson",
"NodeSetPropertiesFromJson",
"NodeSetProperty",
"NodeSetPropertyFromJson",
"RelationshipAdd",
"RelationshipDeleteProperties",
"RelationshipDeleteProperty",
"RelationshipRemove",
"RelationshipResetPropertiesFromJson",
"RelationshipSetPropertiesFromJson",
"RelationshipSetProperty",
"RelationshipSetPropertyFromJson"
};
…and a third that contains all the read operations to the graph which we wont put here.
Now when we build our environment we will pass in a Permission. Notice there is no “break” in our switch statement so if we pass in ADMIN to our switch statement we get all 3, if we pass WRITE, we get the bottom two:
void Sandbox::buildEnvironment(Permission permission) {
...
switch (permission) {
case Permission::ADMIN: {
copyAll(env, lua.globals(), ALLOWED_GRAPH_ADMIN_FUNCTIONS);
}
case Permission::WRITE: {
copyAll(env, lua.globals(), ALLOWED_GRAPH_WRITE_FUNCTIONS);
}
default: {
copyAll(env, lua.globals(), ALLOWED_GRAPH_READ_FUNCTIONS);
}
}
Back in our Shard.h, we’ll add two more environments to hold these different sets of permissions:
sol::environment env; // Lua Sandboxed Environment
sol::environment read_write_env; // Lua Read Write Sandboxed Environment
sol::environment read_only_env; // Lua Read Only Sandboxed Environment
We’ll create them when we initialize the shard:
auto sandbox = Sandbox(lua, Permission::ADMIN);
env = sandbox.getEnvironment();
auto rw_sandbox = Sandbox(lua, Permission::WRITE);
read_write_env = rw_sandbox.getEnvironment();
auto ro_sandbox = Sandbox(lua, Permission::READ);
read_only_env = ro_sandbox.getEnvironment();
To wire these environments together to our server, we’ll put a wrapper around our our RunLua function that takes the environment in to account:
seastar::future<std::string> Shard::RunAdminLua(const std::string &script){
return RunLua(script, env);
}
seastar::future<std::string> Shard::RunRWLua(const std::string &script) {
return RunLua(script, read_write_env);
}
seastar::future<std::string> Shard::RunROLua(const std::string &script) {
return RunLua(script, read_only_env);
}
On our http handler we’ll modify the URL prefix to be “db” for admin, “rw” for read write and “ro” for read only.
void Lua::set_routes(routes &routes) {
auto postLua = new match_rule(&postLuaHandler);
postLua->add_str("/db/" + graph.GetName() + "/lua");
routes.add(postLua, operation_type::POST);
auto postLuaRW = new match_rule(&postLuaRWHandler);
postLuaRW->add_str("/rw/" + graph.GetName() + "/lua");
routes.add(postLuaRW, operation_type::POST);
auto postLuaRO = new match_rule(&postLuaROHandler);
postLuaRO->add_str("/ro/" + graph.GetName() + "/lua");
routes.add(postLuaRO, operation_type::POST);
}
To change the url prefix we send the script to in our UI, we’ll modify our javascript to keep an eye on a radio button in our Settings panel that controls the permissions:
let permissions = "db";
let radios = document.querySelectorAll('input[type=radio][name="permissions"]');
radios.forEach(radio => radio.addEventListener('change', () => permissions = radio.value));
async function sendscript() {
...
let sel = document.getElementById("databases");
if (sel.options.length > 0) {
let selected_database = sel.options[0].value;
if (sel.selectedIndex > -1) {
selected_database = sel.options[sel.selectedIndex].value;
}
let url = '/' + permissions + '/' + selected_database + '/lua';
You’ll also notice we’re also grabbing the selected database we’re going to send our script to. That’s because our settings panel includes this selection as well. Here is what it looks like:
In our settings panel we can:
- choose which database we want to interact with
- create a new database
- reset an existing database
- delete an existing database
I decided to make Create, Reset and Delete all take a separate input text of the name of the database to serve as a safety against accidentally clicking the button in error because I’ve been there and I want to prevent you from going there.
A database is basically a Graph and the http handlers that connect it to our http server. So let’s build a new class for this:
class Database {
public:
explicit Database(std::string name): graph(name), healthCheck(graph), schema(graph),
nodes(graph), relationships(graph), nodeProperties(graph), relationshipProperties(graph),
degrees(graph), neighbors(graph), connected(graph), lua(graph), restore(graph) {}
ragedb::Graph graph;
HealthCheck healthCheck;
Schema schema;
Nodes nodes;
Relationships relationships;
NodeProperties nodeProperties;
RelationshipProperties relationshipProperties;
Degrees degrees;
Neighbors neighbors;
Connected connected;
Lua lua;
Restore restore;
The database objects we’re going to create need a container to hang out in. This container class will create, destroy, reset, stop, retrieve and check if a database exists. It also needs to point to the server so it can register those http handlers. Where do you put a bunch of database objects? In a Databases class of course:
class Databases {
std::map<std::string, Database> databases;
http_server_control* server;
public:
explicit Databases(http_server_control*& _server) : server(_server) {}
std::vector<std::string> list();
std::string get(std::string key);
bool contains(std::string key);
seastar::future<bool> add(std::string key);
Database &at(std::string key);
seastar::future<bool> reset(std::string key);
seastar::future<bool> remove(std::string key);
seastar::future<bool> stop();
Most of these methods basically call the corresponding future for the database key passed in. The interesting one is add since it needs to wire the handlers. First we check to see if a database by the same name already exists which we can’t allow. Then we create the database by emplacing it on our databases map. We need to wire in the routes, so we create a vector of futures with all of these set_route calls, and start the actual graph. Once all the futures are done, we return triumphantly:
seastar::future<bool> Databases::add(std::string key) {
if (databases.contains(key)) {
return seastar::make_ready_future<bool>(false);
}
databases.emplace(key, key);
auto &database = databases.at(key);
std::vector<seastar::future<>> futures;
futures.emplace_back(server->set_routes([&database](routes &r) { database.relationshipProperties.set_routes(r); }));
futures.emplace_back(server->set_routes([&database](routes &r) { database.nodeProperties.set_routes(r); }));
futures.emplace_back(server->set_routes([&database](routes &r) { database.degrees.set_routes(r); }));
futures.emplace_back(server->set_routes([&database](routes &r) { database.neighbors.set_routes(r); }));
futures.emplace_back(server->set_routes([&database](routes &r) { database.connected.set_routes(r); }));
futures.emplace_back(server->set_routes([&database](routes &r) { database.relationships.set_routes(r); }));
futures.emplace_back(server->set_routes([&database](routes &r) { database.nodes.set_routes(r); }));
futures.emplace_back(server->set_routes([&database](routes &r) { database.schema.set_routes(r); }));
futures.emplace_back(server->set_routes([&database](routes &r) { database.lua.set_routes(r); }));
futures.emplace_back(server->set_routes([&database](routes &r) { database.healthCheck.set_routes(r); }));
futures.emplace_back(server->set_routes([&database](routes &r) { database.restore.set_routes(r); }));
futures.emplace_back(databases.at(key).start());
auto p = make_shared(std::move(futures));
return seastar::when_all_succeed(p->begin(), p->end()).then([p] () {
return seastar::make_ready_future<bool>(true);
});
}
Great, so let’s add a Management handler and some methods. I’ll just show you a few bits. The handler has a reference to the databases we created earlier and routes for wiring in those create, reset, delete, get and list database(s) methods:
class Management {
class GetDatabasesHandler : public httpd::handler_base {
public:
explicit GetDatabasesHandler(Management & management) : parent(management) {};
private:
Management & parent;
future<std::unique_ptr<reply>> handle(const sstring& path, std::unique_ptr<request> req, std::unique_ptr<reply> rep) override;
};
...
private:
Databases &databases;
GetDatabasesHandler getDatabasesHandler;
GetDatabaseHandler getDatabaseHandler;
PostDatabaseHandler postDatabaseHandler;
PutDatabaseHandler putDatabaseHandler;
DeleteDatabaseHandler deleteDatabaseHandler;
public:
Management(Databases &_databases) : databases(_databases), getDatabasesHandler(*this), getDatabaseHandler(*this),
postDatabaseHandler(*this), putDatabaseHandler(*this), deleteDatabaseHandler(*this) {}
void set_routes(routes& routes);
Let’s take a deeper dive into the post database handle which creates new databases. It lives at the “/dbs/{key}” url of our server and takes a POST request:
void Management::set_routes(routes &routes) {
...
auto postDatabase = new match_rule(&postDatabaseHandler);
postDatabase->add_str("/dbs");
postDatabase->add_param("key");
routes.add(postDatabase, operation_type::POST);
When it gets a request, it checks the key parameter to make sure it is valid and then calls add method of the databases class we created earlier with that key. If it’s all good, we return the json value of the database.
future<std::unique_ptr<reply>> Management::PostDatabaseHandler::handle([[maybe_unused]] const sstring &path, std::unique_ptr<request> req, std::unique_ptr<reply> rep) {
bool valid_key = Utilities::validate_parameter(Utilities::KEY, req, rep, "Invalid key");
if (valid_key) {
std::string key = req->param[Utilities::KEY];
return parent.databases.add(key).then([key, req = std::move(req), rep = std::move(rep), this] (bool success) mutable {
if (success) {
rep->write_body("json", sstring(parent.databases.get(key)));
parent.databases.at(key).graph.Log(req->_method, req->get_url());
} else {
rep->write_body("json", json::stream_object("Database already exists"));
rep->set_status(reply::status_type::bad_request);
}
return make_ready_future<std::unique_ptr<reply>>(std::move(rep));
});
}
return make_ready_future<std::unique_ptr<reply>>(std::move(rep));
}
If there is an error we, alert the user in our javascript:
That’s all there is to it. I was pleasantly surprised by how little code it took to add both permissions and multiple databases to RageDB. There is of course a lot more to do, but we keep marching on.
On another topic, I have a new employer. I’ve joined the team at Relational.ai
Take a look at the video below to get an idea of what the team is building.
It is 180 degrees from RageDB. It’s Relational, it’s declarative, it’s cloud native, it’s meant for analytics and data science. It is incredibly ambitious and a significant forward step in the field of databases…and you can help. Take a look at the current job openings and join the team. Tell your friends. If you know someone who would be perfect for a role, tell them to come join us.
Do you want to write SQL queries for the rest of your life, or do you want to come with me and change the world?
Max De Marzi 