Async Concurrency Control in Node.js: Promise Pools, Backpressure, and Cancellation
Fire off 10,000 concurrent API calls and you get rate-limited, OOM-killed, or both. Batch them with Promise.all and one slow item blocks the whole batch. Here is the promise-pool pattern with backpressure and abort signals that gives you full control over concurrent async work.
The batch job started at 02:00. By 02:03, the Postgres connection pool was exhausted, the third-party API had returned 429 for 400 consecutive requests, and the process had consumed 2.1 GB of heap. The code looked innocent enough:
const results = await Promise.all(
records.map(record => apiCall(record.id))
);This is the single most dangerous async pattern in Node.js, and it is everywhere. Promise.all with Array.map creates every promise instantly. If records has 10,000 items, the runtime creates 10,000 in-flight HTTP connections, file handles, or database queries before a single one resolves. The event loop is busy, but your machine is dying.
The fix is a pattern I call a promise pool: a fixed number of concurrent slots that process items from a queue, starting a new operation as soon as one finishes, with backpressure when the producer outpaces the consumer, and graceful cancellation when the caller changes its mind.
Why Promise.all.map is dangerous
The problem is not Promise.all itself. The problem is that mapping over an array and wrapping every element in a promise creates all promises in the same microtick. Node.js does not wait for the first one to resolve before creating the second. It creates all of them, registers all their callbacks, and lets the runtime sort out the mess.
Here is what happens in practice:
Connection pool exhaustion. Postgres, Redis, and HTTP clients all have connection pools. When you create 10,000 concurrent queries, they all try to acquire a connection from a pool of 10. Nine thousand nine hundred ninety queue up, each holding a socket open, consuming memory, and timing out.
Remote API rate limiting. The external API accepts 50 requests per second. Your code sends 10,000 in under a second. The first 50 succeed. The next 9,950 get 429 responses. You are now burning time on retries that you could have avoided by throttling.
OOM from buffered responses. Each in-flight request allocates buffers for headers, body chunks, and response objects. Node.js does not reject a promise because it is waiting in a queue. It holds everything in memory until the connection resolves or times out.
No backpressure mechanism. The producer (reading from a file, iterating a cursor, consuming a queue) has no way to know that the consumer is drowning. It keeps adding items to the array, and the memory grows until the OS kills the process.
The promise pool pattern
A promise pool fixes all of these with a single mechanism: a fixed number of concurrent slots, and a queue of pending work. When a slot opens (a promise resolves or rejects), the next item is dequeued and started.
Here is a minimal, production-ready implementation:
export async function promisePool<T, R>(
items: T[],
mapper: (item: T, signal: AbortSignal) => Promise<R>,
concurrency: number,
signal?: AbortSignal
): Promise<R[]> {
const results: R[] = new Array(items.length);
let nextIndex = 0;
let completed = 0;
if (signal?.aborted) {
throw new DOMException('Aborted', 'AbortError');
}
async function worker(): Promise<void> {
while (nextIndex < items.length && !signal?.aborted) {
const i = nextIndex++;
results[i] = await mapper(items[i], signal ?? new AbortController().signal);
completed++;
}
}
const workers = Array.from({ length: Math.min(concurrency, items.length) }, worker);
await Promise.all(workers);
if (signal?.aborted) {
throw new DOMException('Aborted', 'AbortError');
}
return results;
}This is not a wrapper around p-limit. It uses a shared mutable index (nextIndex) that each worker atomically reads and increments. When a worker finishes one item, it checks whether there are more items and runs the next one. The concurrency limit is the number of workers, not the number of promises created per second.
The key detail is that this does not create all promises upfront. It creates exactly concurrency promises at the start, and each one creates its successor when it resolves. The memory footprint is bounded by the concurrency limit, not the input size.
Using the pool with an abort signal
The signal parameter lets the caller cancel all in-flight work without leaking resources. Here is the pattern:
const ac = new AbortController();
// Start processing
const promise = promisePool(
records,
async (record, signal) => {
const response = await fetch(`https://api.example.com/items/${record.id}`, {
signal, // forward the signal to fetch
});
return response.json();
},
10,
ac.signal
);
// Cancel after 5 seconds if not done
setTimeout(() => ac.abort(), 5000);
try {
const results = await promise;
console.log(`Processed ${results.length} items`);
} catch (err) {
if (err.name === 'AbortError') {
console.log('Operation was cancelled');
} else {
throw err;
}
}The AbortError propagation is critical. Without it, a cancelled operation leaves dangling promises that resolve or reject into the void, holding references and preventing garbage collection. Every mapper function should forward the signal to any underlying async operation (fetch, database query, file read) so the runtime can clean up resources immediately.
Handling rejection without killing the pool
The pool above has a problem: if one item’s mapper rejects, the entire pool throws. That is correct for some use cases (atomic batch operations), but wrong for others (processing a queue where individual failures should not abort everything).
For the latter, wrap the mapper to catch errors:
type Result<T> = { ok: true; value: T } | { ok: false; error: unknown };
async function promisePoolSafe<T, R>(
items: T[],
mapper: (item: T, signal: AbortSignal) => Promise<R>,
concurrency: number,
signal?: AbortSignal
): Promise<Result<R>[]> {
const wrapped = async (item: T, sig: AbortSignal): Promise<Result<R>> => {
try {
const value = await mapper(item, sig);
return { ok: true, value };
} catch (error) {
return { ok: false, error };
}
};
return promisePool(items, wrapped, concurrency, signal);
}Callers can then inspect results:
const results = await promisePoolSafe(records, mapper, 10);
const failures = results.filter(r => !r.ok);
const successes = results.filter(r => r.ok);
if (failures.length > 0) {
logger.warn(`${failures.length} items failed out of ${results.length}`);
}Backpressure: when the producer is faster than the pool
Promise pools bound the consumer, but what about the producer? If you are reading rows from a Postgres cursor or consuming messages from Kafka, the producer may deliver items faster than the pool can process them. Without backpressure, the producer fills an in-memory buffer until the process runs out of memory.
The fix is an async iterator that does not yield the next item until there is capacity in the pool:
class BackpressureQueue<T> {
private items: T[] = [];
private resolvers: ((item: T) => void)[] = [];
push(item: T): void {
if (this.resolvers.length > 0) {
const resolve = this.resolvers.shift()!;
resolve(item);
} else {
this.items.push(item);
}
}
[Symbol.asyncIterator](): AsyncIterator<T> {
return {
next: (): Promise<IteratorResult<T>> => {
if (this.items.length > 0) {
return Promise.resolve({ value: this.items.shift()!, done: false });
}
return new Promise((resolve) => {
this.resolvers.push((item: T) => resolve({ value: item, done: false }));
});
},
};
}
get size(): number {
return this.items.length;
}
}This gives you a bounded queue with backpressure. When the producer pushes faster than the consumer pulls, items accumulate in the buffer. When the buffer hits a limit, the producer blocks on push. The consumer processes at its own pace.
In practice, add a maximum buffer size:
class BoundedQueue<T> extends BackpressureQueue<T> {
private maxSize: number;
constructor(maxSize = 1000) {
super();
this.maxSize = maxSize;
}
async pushAndWait(item: T): Promise<void> {
while (this.size >= this.maxSize) {
await new Promise(resolve => setTimeout(resolve, 10));
}
this.push(item);
}
}This pattern maps directly to database cursors. Instead of loading 100,000 rows into an array and then processing them, you stream rows through the bounded queue:
const queue = new BoundedQueue<Row>(500);
const ac = new AbortController();
// Producer: stream rows from Postgres
const producer = (async () => {
for await (const row of db.stream('SELECT * FROM large_table')) {
if (ac.signal.aborted) break;
await queue.pushAndWait(row);
}
})();
// Consumer: process rows with a pool
const consumer = promisePool(
{ [Symbol.asyncIterator]: () => queue[Symbol.asyncIterator]() } as any,
async (row) => processRow(row),
10,
ac.signal
);
try {
await Promise.all([producer, consumer]);
} catch {
ac.abort();
}Priority: when some items matter more
Not all work is equal. In a real-time dashboard, rendering the visible chart data is more important than prefetching last month’s analytics. A flat pool treats them the same.
The fix is a priority queue with a pool on top:
interface PrioritizedItem<T> {
item: T;
priority: number; // higher = more urgent
}
async function priorityPool<T, R>(
items: PrioritizedItem<T>[],
mapper: (item: T, signal: AbortSignal) => Promise<R>,
concurrency: number,
signal?: AbortSignal
): Promise<R[]> {
const sorted = [...items].sort((a, b) => b.priority - a.priority);
return promisePool(
sorted.map(p => p.item),
mapper,
concurrency,
signal
);
}This works for a static batch. For a dynamic stream where priorities change, you need a priority queue data structure (a binary heap) and the ability to preempt in-flight work by aborting low-priority items and restarting them later. That is more complex and usually worth it only when low-priority items take minutes and block high-priority ones.
real-world example: crawling an API with rate-limit awareness
Here is the pattern in practice: crawling a paginated API that rate-limits to 50 requests per second, with automatic retry and backpressure.
import { promisePool } from './promise-pool';
interface Page {
page: number;
items: unknown[];
}
async function crawlApi(baseUrl: string, totalPages: number): Promise<unknown[]> {
const ac = new AbortController();
const allItems: unknown[] = [];
const pages: number[] = Array.from({ length: totalPages }, (_, i) => i + 1);
const mappers = async (page: number, signal: AbortSignal): Promise<Page> => {
const url = `${baseUrl}?page=${page}&per_page=100`;
const response = await fetch(url, { signal });
if (response.status === 429) {
// Rate-limited: wait and retry exactly once
const retryAfter = parseInt(
response.headers.get('retry-after') ?? '1',
10
);
await new Promise(resolve => setTimeout(resolve, retryAfter * 1000));
const retryResponse = await fetch(url, { signal });
if (!retryResponse.ok) {
throw new Error(`Failed to fetch page ${page}: ${retryResponse.status}`);
}
return { page, items: await retryResponse.json() };
}
if (!response.ok) {
throw new Error(`Failed to fetch page ${page}: ${response.status}`);
}
return { page, items: await response.json() };
};
// Concurrency of 5 gives roughly 5 parallel requests,
// well under the 50/sec rate limit
const results = await promisePool(pages, mappers, 5, ac.signal);
for (const { items } of results) {
allItems.push(...items);
}
return allItems;
}The concurrency of 5 ensures that even with retries, the total request rate stays under the limit. The AbortSignal propagates to fetch, so if the caller decides to cancel mid-crawl, all in-flight HTTP connections are torn down immediately.
When you should not use a pool
Promise pools are not always the right tool.
CPU-bound work should use worker threads, not a promise pool. A promise pool running CPU-intensive operations just parallelizes the waiting on the event loop. The actual CPU work still runs on one thread. Use worker_threads or child_process for that.
Very low concurrency (2-3) does not need a pool. If you are calling two external APIs in parallel, just use Promise.all([api1(), api2()]). The overhead of a pool is not justified.
Streams are sometimes a better abstraction. If you are processing a file line by line, a Transform stream with highWaterMark is a more native way to manage backpressure than a queue. Use the right tool for the data shape.
Performance characteristics
The promise pool adds near-zero overhead per item. The cost is one function call and one index increment per item, plus the promise machinery around each mapper invocation. For I/O-bound work (HTTP calls, database queries, file reads), the pool overhead is in the microsecond range, dwarfed by the milliseconds of I/O latency.
Memory is bounded by concurrency * max(per-item memory). If each item holds a 1 MB response and concurrency is 10, worst-case memory is 10 MB, regardless of whether the input has 100 or 100,000 items.
The practical takeaway
The promise pool pattern is one of the few async primitives I reach for in almost every Node.js backend. It is the default alternative to Promise.all(arr.map(fn)), and it prevents the three most common production failures: connection pool exhaustion, unintentional DDoS against remote APIs, and OOM kills from unbounded in-flight work.
Before your next batch job or data pipeline, run through this checklist:
- Are all promises created in a single microtick? If yes, a pool is needed.
- Is the concurrency bound to a number that matches your connection pool size, rate limit, or memory budget?
- Does every mapper accept and forward an
AbortSignal? - Is the caller prepared to handle
AbortErrorcleanly? - If individual failures should not abort the batch, are you using a result wrapper type?
- Is the producer throttled or is it filling an unbounded buffer? If the latter, add backpressure.
- Does the pool outlive the items it processes? If the Node.js process stays alive afterward, make sure all references are released.
The promise pool does not make your async code faster in the throughput sense. It makes it predictable. Predictable memory, predictable connection usage, predictable latency. That predictability is what keeps a batch job running at 3 AM instead of waking you up with an OOM alert.
A note from Yojji
Writing production-grade async code is not about clever one-liners or the newest stream library. It is about understanding the resource boundaries of your runtime and building backpressure into every layer of your application. That is the engineering discipline that separates reliable batch processing from late-night pager alerts.
Yojji is an international custom software development company founded in 2016, with offices in Europe, the US, and the UK. Their senior engineering teams specialize in the JavaScript ecosystem (React, Node.js, TypeScript), cloud platforms (AWS, Azure, Google Cloud), and full-cycle product delivery from discovery through DevOps and production operations.