Dragonfly Cloud announces new enterprise security features - learn more

Dragonfly

Question: How can one implement in-memory caching with MongoDB?

Answer

To implement in-memory caching with MongoDB, you would typically use the MongoDB's WiredTiger storage engine which supports an in-memory storage engine. The in-memory storage engine allows for all data to be stored in memory, providing a solution where read latency must be minimized.

Here is an example of running MongoDB with the in-memory storage engine using Docker:

docker run --name mongo-in-memory -d mongo --storageEngine=inMemory

Or if you are starting MongoDB from command line:

mongod --storageEngine inMemory

Remember, though, that because the in-memory storage engine does not persist data, when the mongod process terminates, all data is lost.

It’s also important to note that while MongoDB has this built-in support for in-memory storage, it doesn't replace other kinds of external caching systems like Redis. MongoDB's in-memory option is best used for specific use-cases where the persistence of the data isn't required (e.g., caching session data).

For cases where you need to cache frequently accessed DB results, an external caching layer such as Redis is often used in combination with MongoDB, keeping the frequently accessed data in fast, volatile storage for quick retrieval, and storing other data in MongoDB.

An example of using Node.js, MongoDB, and Redis together looks something like this:

const express = require('express');
const axios = require('axios');
const redis = require('redis');
const mongoose = require('mongoose');

// setup Express App
const app = express();

// create and connect to a Redis client
const client = redis.createClient();

client.on('connect', () => console.log('Connected to Redis'));

// connect to MongoDB
mongoose.connect('mongodb://localhost/test', { useNewUrlParser: true });
const db = mongoose.connection;

app.get('/data', (req, res) => {
  const dataKey = 'your-data-key';
  
  // Try fetching the result from Redis first in case we have it cached
  return client.get(dataKey, async (err, data) => {
    if (data) {
      return res.json({ source: 'cache', data: JSON.parse(data) })
    } 
    else {
      // Get the data from MongoDB and store to Redis
      const dataFromMongoDB = await db.collection('yourCollection').findOne({});
      client.setex(dataKey, 3600, JSON.stringify(dataFromMongoDB));
      
      return res.json({ source: 'MongoDB', data: dataFromMongoDB });
    }
  });
});

app.listen(3000, () => console.log('Server is running on port 3000'));

So, while MongoDB can support in-memory operations, a multi-layered caching strategy including MongoDB for persistence storage and Redis for high-speed, in-memory caching yields optimal results in most scenarios.

Was this content helpful?

Other Common In Memory Questions (and Answers)

White Paper

Free System Design on AWS E-Book

Download this early release of O'Reilly's latest cloud infrastructure e-book: System Design on AWS.

Free System Design on AWS E-Book

Switch & save up to 80% 

Dragonfly is fully compatible with the Redis ecosystem and requires no code changes to implement. Instantly experience up to a 25X boost in performance and 80% reduction in cost