Sharing an Outbound Rate Limit Across Node.js Workers

While building a WhatsApp integration for a client's CRM, I kept running into outbound rate limits. Getting one Node.js process to slow down was fairly straightforward. Getting several processes to agree on the same limit took more thought.

Say an API allows 80 requests a second. You put an in-memory limiter in your backend, set it to 80, and it works. Then you run four instances. Each one stays under 80, but together they can send 320 requests a second using the same credential. The provider still only allows you 80.

So the counter has to be shared. And when the provider tells one worker to wait, the others need to hear about that too.

Inbound is the easier one

An inbound limiter handles requests coming into your server. You add something like express-rate-limit, and requests over the limit get a 429.

Outbound is where you're the one making too many calls. If those calls are messages a customer asked you to send, you probably can't just drop them. They need to wait somewhere until there's room, with a limit on how long you're willing to keep them waiting.

This can happen between your own services too. A queue lets a consumer work at its own pace, and a circuit breaker stops calls to a failing service. But you still have to enforce the requests-per-second limit yourself.

What a campaign does to this

For the examples here, I'll use 80 messages a second per business phone number. Meta's throughput documentation also describes higher throughput, up to 1,000 messages a second, for eligible numbers. Use whatever limit applies to your number.

A tenant starts a campaign, and suddenly there's a list of messages to send. If every message becomes an immediate API call, you go over the limit and start getting errors like 130429. Now you also have to work out which recipients still need their message.

With throttling, the burst stays in your queue. Workers send at the rate the phone number's budget allows. The campaign takes time to finish, but you're not making calls you already know will fail.

Meta also counts inbound traffic toward that throughput, and there's a separate pair limit for repeated messages to one recipient. Your outgoing counter can't see all of that, so you'll still need to handle rate-limit responses.

Start with one process

Keep a counter in memory

The simplest version counts calls in the current window. If there's room, send. Otherwise, wait for the window to reset:

const LIMIT = 80;          // calls allowed
const WINDOW_MS = 1000;    // per second
 
let windowStart = Date.now();
let count = 0;
 
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
 
async function acquire() {
  while (true) {
    const now = Date.now();
    if (now - windowStart >= WINDOW_MS) {
      windowStart = now;
      count = 0;
    }
    if (count < LIMIT) {
      count++;
      return;
    }
    // wait for the current window to end, then check again
    await sleep(WINDOW_MS - (now - windowStart));
  }
}
 
async function sendWhatsAppMessage(payload) {
  await acquire();
  return fetch(META_URL, { method: "POST", body: JSON.stringify(payload) /* ... */ });
}

The check and increment both happen before an await, so within one Node.js isolate, another caller can't slip in between them.

The catch is the window boundary. You can send 80 calls just before the reset and another 80 just after it, putting 160 calls within a few milliseconds. Some providers allow that. Others measure the window differently.

Space the calls out

If that burst is a problem, give each call a time slot after the previous one:

class Throttle {
  constructor(perSecond) {
    this.gap = 1000 / perSecond;
    this.nextSlot = 0;
  }
 
  async acquire() {
    const now = Date.now();
    const slot = Math.max(now, this.nextSlot);
    this.nextSlot = slot + this.gap; // reserve the slot before awaiting
    const wait = slot - now;
    if (wait > 0) await new Promise((r) => setTimeout(r, wait));
  }
}
 
const metaThrottle = new Throttle(80);
await metaThrottle.acquire();

Each caller reserves its slot before sleeping. At 80 calls a second, that's a slot every 12.5 milliseconds, a simple form of leaky-bucket scheduling.

There are still limits to this little class. Node.js timers aren't exact, and a stalled event loop can wake several callers together. There's also nothing here to bound the queue or cancel a call that's waiting.

If you want to allow a small burst, a token bucket gives you that. Tokens refill at a fixed rate up to a capacity, and every call spends one. When the bucket is empty, you wait.

Or use a library

You don't have to write the queue yourself. p-queue has concurrency and interval caps. rate-limiter-flexible has a memory limiter with a queue wrapper. Bottleneck gives you minTime, maxConcurrent, and reservoirs.

Here's a bounded local queue with rate-limiter-flexible:

import { RateLimiterMemory, RateLimiterQueue } from "rate-limiter-flexible";
 
const limiter = new RateLimiterMemory({ points: 80, duration: 1 });
const queue = new RateLimiterQueue(limiter, { maxQueueSize: 10_000 });
 
await queue.removeTokens(1); // resolves when it's our turn
await callMeta();

All of this works within one process. The counters still live in that process's memory.

What changes when you add workers

Separate containers don't share that memory. Neither do PM2 cluster workers, Node.js cluster processes, or a background worker running next to your API. They can all call the same provider without knowing what the others have sent.

You could divide 80 by the number of instances. Four instances get 20 each. But then you scale to six and you're allowing 120 again. Scale down to two and you're using only half the budget.

Even with four instances, the work probably won't be spread evenly. One might have 500 messages queued for a phone number while the other three have none. That busy instance is stuck at 20 calls a second, with the rest of the number's budget sitting unused.

So the limit needs to follow the credential or business phone number, wherever the calls come from. Every process sending for that identity has to check the same state.

Move the budget to Redis

Redis gives all the workers somewhere to check before sending. You still have a few ways to keep the count:

• A fixed window uses an atomic INCR and an expiry. The Redis counter pattern covers this approach. Updating the counter and setting its expiry must not leave a persistent key if a caller fails between commands.
• A sliding-window log keeps call timestamps in a sorted set, removes expired entries, and counts the remainder. It avoids aligned-window boundaries at the cost of storing an entry per call. Weighted counters offer an approximation with less state.
• A token bucket tracks available tokens and elapsed refill time. Reading, refilling, spending, and saving must happen together, usually in a Lua script.

This script reserves a token and tells the caller how long to wait. The balance can go negative to account for calls already waiting. It uses Redis's clock, so a worker with a slightly different system clock won't change the calculation.

-- KEYS[1]   bucket key, e.g. "rl:meta:<PHONE_NUMBER_ID>"
-- ARGV[1]   capacity (max burst)
-- ARGV[2]   refill rate, tokens per second
-- ARGV[3]   max wait in ms before we give up
local capacity = tonumber(ARGV[1])
local rate     = tonumber(ARGV[2]) / 1000   -- tokens per ms
local max_wait = tonumber(ARGV[3])
 
-- use Redis's clock, not the caller's, so instance clock skew doesn't matter
local t   = redis.call('TIME')
local now = tonumber(t[1]) * 1000 + math.floor(tonumber(t[2]) / 1000)
 
local state  = redis.call('HMGET', KEYS[1], 'tokens', 'ts')
local tokens = tonumber(state[1]) or capacity
local ts     = tonumber(state[2]) or now
 
tokens = math.min(capacity, tokens + (now - ts) * rate)
tokens = tokens - 1   -- reserve a token, even if that takes us below zero
 
local wait = 0
if tokens < 0 then
  wait = math.ceil(-tokens / rate)
  if wait > max_wait then
    return -1          -- backlog too long, let the caller decide
  end
end
 
redis.call('HSET', KEYS[1], 'tokens', tokens, 'ts', now)
redis.call('PEXPIRE', KEYS[1], math.ceil(capacity / rate) + max_wait)
return wait

If the wait would exceed max_wait, the script returns -1 without saving that reservation. Otherwise, the caller waits for its slot. On the Node.js side, using ioredis:

import Redis from "ioredis";
import { readFileSync } from "node:fs";
 
const redis = new Redis(process.env.REDIS_URL);
redis.defineCommand("reserveToken", {
  numberOfKeys: 1,
  lua: readFileSync("./token-bucket.lua", "utf8"),
});
 
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
 
export async function acquireMetaSlot(phoneNumberId) {
  const wait = await redis.reserveToken(
    `rl:meta:${phoneNumberId}`,
    80,      // capacity
    80,      // tokens per second
    30_000,  // give up if the wait is over 30s
  );
  if (wait < 0) throw new Error("Rate limit backlog too long");
  if (wait > 0) await sleep(wait);
}

With these settings, the bucket starts with 80 tokens and refills at 80 a second. That means it allows an initial burst, so you can still exceed 80 calls in a rolling second. If the provider requires stricter spacing, you'll need a smaller burst allowance or a different algorithm.

Every worker using the same phone-number key now reserves from the same bucket. Adding workers doesn't create more capacity. You do still have to deal with cancelled calls wasting a reservation and delayed callers sending after their slot.

rate-limiter-flexible also offers RateLimiterRedis. Its RateLimiterQueue still waits inside each process, while the backing limiter shares the count.

Redis is now part of every send

If Redis goes down, what happens to the messages? You can stop sending, leave them waiting, or fall back to a local limiter. But once you fall back locally, the workers no longer share a limit. That's a decision to make before it happens.

There's also a store round trip for every call. And if you're waiting inside the original API request, that HTTP connection stays open the whole time. A campaign that takes minutes to drain needs somewhere else to wait.

GCRA, the generic cell rate algorithm, is another useful option. It represents pacing with a timestamp instead of a log of calls. Brandur Leach's GCRA explanation covers the mechanism; Stripe's rate-limiter article discusses several production concerns from the inbound side.

Let the campaign wait on a queue

For something like a campaign, I'd put the work on a durable queue and return an accepted response. Workers send the messages, and the CRM shows progress using stored statuses or webhooks. The person who clicked Send doesn't need to keep a request open until the last message goes out.

BullMQ's rate limiter is shared across workers on the same queue:

import { Worker } from "bullmq";
 
const worker = new Worker(
  "whatsapp-outbound",
  async (job) => sendToMeta(job.data),
  {
    connection,
    limiter: { max: 80, duration: 1000 }, // 80 jobs/sec across ALL workers
  },
);

Adding workers to this queue won't multiply the 80-jobs-per-second allowance. Just keep in mind that jobs and API calls aren't necessarily the same thing. If a job sends a batch or retries a call internally, the queue limiter doesn't count those extra calls for you.

The backlog still takes storage, so you need retention rules and a point at which an old message is too late to send. But it no longer needs an open HTTP connection for every waiting request.

The catch for a multi-tenant app is that this limit applies to the whole queue. BullMQ Pro group limits let you keep separate budgets within it. You can also use one queue per phone number, or keep a shared outbound limiter inside the workers. In that last setup, the queue keeps the jobs alive and the limiter decides when each API call can go out.

A couple of other options

Bottleneck can also share state through Redis, as long as the instances use the same limiter id and a Redis-backed datastore. I'd check its release and issue history against the runtime you're using before taking it on.

You can also put all calls to Meta behind one service. That service owns the credentials, rate limit, and retries, and everything else calls it. It's another service to run and recover, but you get one place to manage the integration. If you already use Envoy, its global rate limit service can be part of that setup.

Share the cooldown too

You can have all your workers agreeing and still get rate limited. Another app might use the same credential, incoming traffic might count toward the allowance, or the provider might change the limit.

When one worker gets Retry-After, the others need to wait too. If only that worker sleeps, the rest keep sending against the same exhausted budget.

Without an explicit retry time, use bounded exponential backoff with jitter. AWS's backoff and jitter article explains how synchronized retries create another burst. Also check whether repeating the request is safe: retrying a message send after an ambiguous response can send the message twice.

For BullMQ, the documented manual rate-limit pattern combines worker.rateLimit(ms) with throw Worker.RateLimitError(). The worker also needs its limiter options configured. This returns the job to waiting and pauses admissions for the queue.

Leaving headroom helps when the provider counts traffic you cannot observe. Sending at 70 against a budget of 80 leaves room for that traffic, though it still cannot guarantee you won't see a rate-limit response.

How this became valve

I ended up building valve around this, published on npm as valvejs. There's an in-memory store if you only have one process, and Redis or Valkey when several workers need to share the budget.

It has GCRA pacing and fixed windows, and can check several rules together. So a message can need room in both the phone number's shared budget and that recipient's budget before it goes out. A blocked recipient doesn't have to hold up work for someone else.

The cooldown is shared too. wrapFetch can retry rate-limit responses, though POST and PATCH need an explicit opt-in after you've checked that repeating the call is safe. Concurrency can stay local or use renewable store leases across workers. If a job loses its lease, it has to honor the cancellation signal.

The jobs waiting inside valve still live in memory. For a campaign that needs to survive a worker restart, I'd keep them in a durable queue and put valve around the actual send. That's the part I wanted the library to handle: several workers making calls through the same account without each inventing its own version of the limit.

References

• Meta: WhatsApp Cloud API throughput
• Meta: WhatsApp Cloud API rate limits
• Redis: counter and rate-limiter patterns
• Redis: scripting with Lua
• BullMQ: rate limiting
• BullMQ Pro: group rate limiting
• valve: source and guarantees