Building a Scalable Job Queue With BullMQ and Redis in Node.js

As applications grow, background processing becomes essential for handling tasks like image processing, email delivery, and data imports. BullMQ is a modern, robust job queue built on top of Redis. Let’s walk through creating a scalable queue system using BullMQ in Node.js.

Why BullMQ?

  • Built on Redis Streams (more scalable than older Bull versions)
  • Supports job retries, rate limiting, events, and sandboxed workers
  • Excellent performance for high-throughput apps

Step 1: Set Up Your Project

npm init -y
npm install bullmq ioredis

Step 2: Create a Queue

// queue.js
const { Queue } = require('bullmq');
const connection = { host: '127.0.0.1', port: 6379 };

const emailQueue = new Queue('emails', { connection });

async function addEmailJob(to, subject, body) {
  await emailQueue.add('send', { to, subject, body });
}

module.exports = { addEmailJob };

Step 3: Set Up a Worker

// worker.js
const { Worker } = require('bullmq');
const connection = { host: '127.0.0.1', port: 6379 };

const worker = new Worker('emails', async job => {
  if (job.name === 'send') {
    console.log(`Sending email to ${job.data.to}...`);
    // Simulate sending
    await new Promise(resolve => setTimeout(resolve, 1000));
    console.log('Email sent!');
  }
}, { connection });

Step 4: Trigger Jobs

// index.js
const { addEmailJob } = require('./queue');

addEmailJob('user@example.com', 'Welcome!', 'Thanks for signing up!');

Step 5: Monitoring and Scaling

  • Use bull-board or arena for a UI dashboard.
  • Scale by running multiple workers or deploying on different machines.
  • Consider retry strategies, job priorities, and concurrency settings.

Conclusion

BullMQ offers a powerful toolkit for background jobs in Node.js. With Redis under the hood and modern features like sandboxed workers, it’s a great choice for any production system requiring asynchronous processing.

If this post helped you, consider supporting me: buymeacoffee.com/hexshift