Redis

In-memory databases are different from the previous two types, as they're usually not structured, which means you have no tables. What you have is normally lists of some kind that you can look up and manipulate, or simple hash tables.

Taking advantage of the Redis instance we installed previously for Hydra, let's see another drawback, or actually feature, of this kind of database. Let's connect to our Redis instance and make the following sequence of instructions:

What we did here was to:

  1. Connect to the Redis service using redis-cli.
  2. Get the content of the counter, which is nil (nothing), because we haven't defined it yet.
  3. Increment the counter, which is now automatically defined and set to 1.
  4. Increment the counter again, which is now 2.
  5. Get the content of the counter, which is of course 2.
  6. Shut down the Redis service.
  7. Start the Redis service.
  8. Connect to the Redis service again.
  9. Get the content of the counter, which is nil (nothing).

Where's our counter? Well, this is an in-memory database, so everything is gone when we shut down the Redis service. This is the design of almost all kinds of in-memory databases.

They're designed to be fast and in-memory. Their purpose is normally to cache data that is expensive to get, such as some complex calculations, or extensive to download, and we want that to be available faster (in-memory).

I wasn't completely fair with Redis as it actually allows your data to be saved between service restarts. So, let's see how far we can go in using it to save our microservice state.

As before, let's uninstall rethinkdb and install the redis module:

npm uninstall rethinkdb --save
npm install redis --save

Let's ignore our settings.json file (you can remove it if you prefer) and assume Redis will be on our local machine.

First, we need to include the redis module and create a Client instance:

const redis = require("redis");
const db = redis.createClient();

We then need to wait until it connects:

db.on("connect", () => {
console.log("db: ready");

// ...
// the rest of our service code
// ...

app.listen(3000, () => {
console.log("app: ready");
});
});

There are a couple of ways we can use Redis to store our data. To make it simple, as we don't have tables, let's use hashes to store our images. Each image will have a different hash, and the name of the hash will be the name of the image.

As there are no tables in this kind of database, our initialization code can just be removed.

Next, let's change our upload method to store data on Redis. As I mentioned, let's store it in a hash with the name of the image:

app.post("/uploads/:name", bodyparser.raw({
limit : "10mb",
type : "image/*"
}), (req, res) => {
db.hmset(req.params.name, {
size : req.body.length,
data : req.body.toString("base64"),
}, (err) => {
if (err) {
return res.send({ status : "error", code: err.code });
}

res.send({ status : "ok", size: req.body.length });
});
});

The hmset command lets us set multiple fields of a hash, in our case, size and data. Notice we're storing our image content in base64 encoding, otherwise we'll lose data. If we restart our service and try to upload our test image, it should work fine:

We can then use redis-cli and see whether our image is there. Well, we're checking to see whether our hash has the field size and matches our image size:

Great! We can now change our Express parameter to look for the image hash:

app.param("image", (req, res, next, name) => {
if (!name.match(/.(png|jpg)$/i)) {
return res.status(403).end();
}

db.hgetall(name, (err, image) => {
if (err || !image) return res.status(404).end();

req.image = image;
req.image.name = name;

return next();
});
});

Our image check method should work now. And, for our download method to work, we just need to change the image loading to decode our previous base64 encoding:

app.get("/uploads/:image", (req, res) => {
let image = sharp(Buffer.from(req.image.data, "base64"));
let width = +req.query.width;
let height = +req.query.height;
let blur = +req.query.blur;
let sharpen = +req.query.sharpen;
let greyscale = [ "y", "yes", "true", "1", "on"].includes(req.query.greyscale);
let flip = [ "y", "yes", "true", "1", "on"].includes(req.query.flip);
let flop = [ "y", "yes", "true", "1", "on"].includes(req.query.flop);

if (width > 0 && height > 0) {
image.ignoreAspectRatio();
}

if (width > 0 || height > 0) {
image.resize(width || null, height || null);
}

if (flip) image.flip();
if (flop) image.flop();
if (blur > 0) image.blur(blur);
if (sharpen > 0) image.sharpen(sharpen);
if (greyscale) image.greyscale();

db.hset(req.image.name, "date_used", Date.now());

res.setHeader("Content-Type", "image/" + path.extname(req.image.name).substr(1));

image.pipe(res);
});

Our images are now being served from Redis. As a bonus, we're adding/updating a date_used field in our image hash to indicate when it was last used:

Removing our image is as simple as removing our hash:

app.delete("/uploads/:image", (req, res) => {
db.del(req.image.name, (err) => {
return res.status(err ? 500 : 200).end();
});
});

We can then try to remove our test image:

Using redis-cli to check whether the hash exists, we see that it's gone:

The only two features missing are the statistics and removing old images.

For the statistics, that could be hard as we're using generic hash tables and we can't be sure how many hash tables are defined, and if all or any have image data. We would have to scan all hash tables, which is complex for large sets.

To remove old images, the problem is the same as there's no way of looking for hash tables with a specific condition, such as a field value.

There are still other paths available to tackle this problem. For example, we could have another hash table with just our image names and use dates. But, the complexity would increase, and the integrity could be at risk as we're splitting information through different hash tables with no certainty of making Atomicity, Consistency, Isolation, and Durability (ACID) operations.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset