The Practical Developer

S3 Presigned Multipart Uploads: Stop Your API Server from Being a File Upload Bottleneck

May 22, 2026 by The Practical Developer

Streaming multi-gigabyte files through your Node.js server burns bandwidth, memory, and connection pools. Here is the direct-to-S3 upload pattern that moves the bytes past your API entirely, with presigned URLs, multipart upload logic, and the security guardrails most tutorials skip.

node.js

infrastructure

performance

Earth viewed from space with glowing network connection lines, symbolizing cloud data flow that bypasses central servers

Your product team wants a feature: users can upload video files up to 5 GB. You wire up a standard multipart form handler in Express, pipe the stream to a temporary disk location, and then upload the whole thing to S3 from your server. The first ten uploads work fine. Then a user on hotel WiFi tries to upload a 2 GB file. The request runs for 14 minutes, ties up one of your event loop threads the whole time, eats 200 MB of memory for the buffer, and fails at 97% because the load balancer idle timeout kills the connection. The user retries. Your server retries the S3 upload. The bill for data transfer out of EC2 to S3 in the same region is small, but the bill for your time is not.

The mistake is architectural, not algorithmic. Your API server should never see the file bytes. The client should upload directly to S3. Your server should only generate a short-lived, scoped URL that lets the client write to a specific key, and then confirm the upload completed. For files under 100 MB, a single presigned PUT URL is enough. For anything larger, you need multipart upload with presigned URLs per part. This post is the production implementation of both, in Node.js, with the security rules that keep random callers from filling your bucket.

Why streaming through your server is wrong

When a client uploads through your API, the data path looks like this:

Client → Load Balancer → Node.js Server → S3

Every byte traverses your server. That means:

  • Memory or disk pressure. Node.js streams to disk if you use multer or busboy, or memory if you buffer. Either way, you are managing resources for someone else’s data.
  • Connection pool exhaustion. The inbound HTTP connection stays open for the entire upload duration. A few slow clients can max out your connection limit.
  • Timeout fragility. Load balancers, reverse proxies, and Node.js itself all have timeouts. The longer the upload, the more likely one of them fires.
  • Double billing. EC2 data transfer to S3 in the same region is free, but if your server is in one AZ and S3 resolves to another, you pay. More importantly, you pay for the EC2 CPU, memory, and connections that do nothing but shuffle bytes.

The fix is to hand the client a temporary write credential to S3 and get out of the way:

Client → Your API  (GET presigned URL)
Client → S3        (PUT bytes directly)
Client → Your API  (POST confirm completion)

Your server sees three small JSON requests instead of a 2 GB payload.

Part 1: presigned PUT for files under 100 MB

For small files, a single presigned PUT URL is the entire solution. The client requests a URL from your API, PUTs the file directly to S3, and tells you the key.

Server: generating the presigned URL

import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';

const s3 = new S3Client({ region: process.env.AWS_REGION });

export async function createPresignedPutUrl(
  key: string,
  contentType: string,
  maxBytes = 100 * 1024 * 1024,
) {
  const command = new PutObjectCommand({
    Bucket: process.env.S3_BUCKET!,
    Key: key,
    ContentType: contentType,
    ContentLength: maxBytes, // S3 will reject if the body exceeds this
  });

  // URL expires in 5 minutes. The upload must start and finish inside this window.
  const url = await getSignedUrl(s3, command, { expiresIn: 300 });
  return url;
}

A few details that matter:

  • ContentType is enforced. If the client PUTs with a different Content-Type header, S3 rejects it. This stops someone from using an image upload endpoint to store executable files.
  • The key should not be client-controlled. The client should request an upload for “my-video.mp4” and your API should generate the final key, for example uploads/2026/05/user-123/${uuid}.mp4. Never let the client pick the S3 key. If they can write to ../../../index.html, you have a path traversal bug in object storage.
  • The expiration should be short. Five minutes is enough for a direct browser upload on a fast connection. If the client needs longer, they can request a fresh URL.

Client: uploading with the presigned URL

async function uploadFile(file: File): Promise<string> {
  const res = await fetch('/api/uploads/small', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ filename: file.name, contentType: file.type }),
  });
  const { url, key } = await res.json();

  const put = await fetch(url, {
    method: 'PUT',
    headers: { 'Content-Type': file.type },
    body: file,
  });
  if (!put.ok) throw new Error(`S3 upload failed: ${put.status}`);

  // Tell your backend the upload is done
  await fetch('/api/uploads/confirm', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ key }),
  });

  return key;
}

This works for files up to the limit of a single PUT request. S3 supports up to 5 GB per PUT, but most browsers and HTTP clients start getting flaky above a few hundred megabytes. If your use case involves files larger than 100 MB, use multipart upload.

Part 2: multipart upload for large files

Multipart upload splits the file into chunks (parts), uploads each part independently, and then tells S3 to assemble them. The client uploads each part directly to S3 using its own presigned URL. Your server only tracks the upload ID and the list of completed part ETags.

Server: initiating the multipart upload

import {
  S3Client,
  CreateMultipartUploadCommand,
  AbortMultipartUploadCommand,
  CompleteMultipartUploadCommand,
  UploadPartCommand,
} from '@aws-sdk/client-s3';
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';

const s3 = new S3Client({ region: process.env.AWS_REGION });

export async function initiateMultipartUpload(
  key: string,
  contentType: string,
) {
  const command = new CreateMultipartUploadCommand({
    Bucket: process.env.S3_BUCKET!,
    Key: key,
    ContentType: contentType,
  });
  const response = await s3.send(command);
  return response.UploadId!;
}

Server: generating presigned URLs for each part

export async function createPresignedPartUrls(
  key: string,
  uploadId: string,
  partCount: number,
) {
  const urls: string[] = [];
  for (let partNumber = 1; partNumber <= partCount; partNumber++) {
    const command = new UploadPartCommand({
      Bucket: process.env.S3_BUCKET!,
      Key: key,
      UploadId: uploadId,
      PartNumber: partNumber,
    });
    const url = await getSignedUrl(s3, command, { expiresIn: 600 });
    urls.push(url);
  }
  return urls;
}

Each URL is scoped to a single part number. The client cannot use the URL for part 3 to upload part 7. S3 validates the part number and upload ID.

Server: completing the multipart upload

export async function completeMultipartUpload(
  key: string,
  uploadId: string,
  parts: { PartNumber: number; ETag: string }[],
) {
  const command = new CompleteMultipartUploadCommand({
    Bucket: process.env.S3_BUCKET!,
    Key: key,
    UploadId: uploadId,
    MultipartUpload: { Parts: parts.sort((a, b) => a.PartNumber - b.PartNumber) },
  });
  await s3.send(command);
}

Client: streaming upload with retry per part

const PART_SIZE = 10 * 1024 * 1024; // 10 MB per part

async function uploadLargeFile(file: File) {
  // 1. Ask the server to start a multipart upload
  const initRes = await fetch('/api/uploads/multipart/init', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ filename: file.name, contentType: file.type }),
  });
  const { key, uploadId, partCount } = await initRes.json();

  // 2. Ask for presigned URLs for every part
  const urlsRes = await fetch('/api/uploads/multipart/urls', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ key, uploadId, partCount }),
  });
  const { urls } = await urlsRes.json();

  // 3. Upload each part directly to S3
  const parts: { PartNumber: number; ETag: string }[] = [];
  for (let i = 0; i < partCount; i++) {
    const start = i * PART_SIZE;
    const end = Math.min(start + PART_SIZE, file.size);
    const chunk = file.slice(start, end);

    const etag = await uploadPartWithRetry(urls[i], chunk, i + 1);
    parts.push({ PartNumber: i + 1, ETag: etag });
  }

  // 4. Tell the server to assemble the parts
  await fetch('/api/uploads/multipart/complete', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ key, uploadId, parts }),
  });

  return key;
}

async function uploadPartWithRetry(url: string, chunk: Blob, partNumber: number, maxAttempts = 3): Promise<string> {
  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    const res = await fetch(url, { method: 'PUT', body: chunk });
    if (res.ok) {
      const etag = res.headers.get('ETag');
      if (!etag) throw new Error(`Missing ETag for part ${partNumber}`);
      return etag;
    }
    if (attempt === maxAttempts) throw new Error(`Part ${partNumber} failed after ${maxAttempts} attempts`);
    await new Promise(r => setTimeout(r, 1000 * attempt));
  }
  throw new Error('unreachable');
}

Note that the client retries individual parts, not the whole upload. If part 12 of 200 fails on flaky WiFi, only that 10 MB chunk is retransmitted. The rest of the 2 GB file is already safely stored in S3 under the upload ID.

The security rules most people skip

Presigned URLs are powerful. A leaked presigned PUT URL is a temporary write credential. You need guardrails.

1. Scope the key, never trust the filename

Never let the client send the S3 key. The client sends metadata (filename, content type). The server generates the key using a fixed prefix and a random ID.

import { randomUUID } from 'node:crypto';

function generateKey(userId: string, filename: string) {
  const safe = filename.replace(/[^a-zA-Z0-9.-]/g, '_');
  return `uploads/${new Date().toISOString().slice(0, 7)}/${userId}/${randomUUID()}-${safe}`;
}

This prevents path traversal, makes logs readable, and groups uploads by month for lifecycle policies.

2. Validate content type against an allowlist

const ALLOWED_TYPES = new Set([
  'video/mp4', 'video/webm', 'image/jpeg', 'image/png', 'application/pdf',
]);

function validateUploadRequest(contentType: string) {
  if (!ALLOWED_TYPES.has(contentType)) {
    throw new Error(`Content type not allowed: ${contentType}`);
  }
}

If you generate a presigned URL for image/png and the client tries to PUT an executable, S3 rejects it automatically. But you should also validate at the API layer so the client gets a clean 400 immediately instead of a confusing S3 XML error later.

3. Enforce a maximum file size

For single PUT uploads, pass ContentLength to the presigned URL command. For multipart, compute the part count from the client-reported size and reject if it exceeds your limit.

const MAX_FILE_SIZE = 5 * 1024 * 1024 * 1024; // 5 GB
const PART_SIZE = 10 * 1024 * 1024; // 10 MB

function validateSize(reportedBytes: number) {
  if (reportedBytes > MAX_FILE_SIZE) throw new Error('File too large');
  if (reportedBytes <= 0) throw new Error('Invalid size');
  return Math.ceil(reportedBytes / PART_SIZE);
}

The client can lie about the size, but the worst case is they get presigned URLs for more parts than they need. The actual S3 object size is bounded by the number of parts you generate times the part size. If they upload less, the final assembled object is smaller. You should still verify the final object size after completion if the exact size matters for downstream processing.

4. Short expiration, tight IAM

The presigned URL expiration should be just long enough for the slowest reasonable upload. For a 10 MB part on 1 Mbps hotel WiFi, that is about 90 seconds. Set the expiration to 10 minutes and move on.

The IAM policy attached to your server should not have s3:*. It should have exactly:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:AbortMultipartUpload",
        "s3:ListMultipartUploadParts",
        "s3:ListBucketMultipartUploads",
        "s3:GetObject"
      ],
      "Resource": [
        "arn:aws:s3:::your-bucket",
        "arn:aws:s3:::your-bucket/uploads/*"
      ]
    }
  ]
}

Notice the bucket policy is scoped to the uploads/* prefix. If your server is compromised, the attacker can only write to the upload prefix, not overwrite your application assets or read sensitive data from other prefixes.

5. Abort incomplete multipart uploads

If a client starts a multipart upload and then closes the browser, S3 keeps the parts forever (or until a lifecycle rule deletes them). Incomplete multipart uploads are invisible in the S3 console unless you know to look for them, and they cost money.

Add a lifecycle rule on the bucket:

{
  "Rules": [
    {
      "ID": "abort-incomplete-multipart",
      "Status": "Enabled",
      "Filter": { "Prefix": "uploads/" },
      "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 1 }
    }
  ]
}

This aborts any multipart upload that is not completed within 24 hours. You can also abort explicitly from your server if the client signals cancellation.

export async function abortMultipartUpload(key: string, uploadId: string) {
  const command = new AbortMultipartUploadCommand({
    Bucket: process.env.S3_BUCKET!,
    Key: key,
    UploadId: uploadId,
  });
  await s3.send(command);
}

Putting it together: the full upload router

// routes/uploads.ts
import { Router } from 'express';
import { randomUUID } from 'node:crypto';

const router = Router();
const ALLOWED_TYPES = new Set(['video/mp4', 'video/webm', 'image/jpeg', 'image/png']);
const MAX_SIZE = 5 * 1024 * 1024 * 1024;
const PART_SIZE = 10 * 1024 * 1024;

router.post('/small', async (req, res) => {
  const { filename, contentType } = req.body;
  if (!ALLOWED_TYPES.has(contentType)) return res.status(400).json({ error: 'bad type' });

  const key = generateKey(req.user.id, filename);
  const url = await createPresignedPutUrl(key, contentType, 100 * 1024 * 1024);
  res.json({ url, key });
});

router.post('/multipart/init', async (req, res) => {
  const { filename, contentType, size } = req.body;
  if (!ALLOWED_TYPES.has(contentType)) return res.status(400).json({ error: 'bad type' });
  if (size > MAX_SIZE) return res.status(400).json({ error: 'too large' });

  const key = generateKey(req.user.id, filename);
  const uploadId = await initiateMultipartUpload(key, contentType);
  const partCount = Math.ceil(size / PART_SIZE);
  res.json({ key, uploadId, partCount });
});

router.post('/multipart/urls', async (req, res) => {
  const { key, uploadId, partCount } = req.body;
  const urls = await createPresignedPartUrls(key, uploadId, partCount);
  res.json({ urls });
});

router.post('/multipart/complete', async (req, res) => {
  const { key, uploadId, parts } = req.body;
  await completeMultipartUpload(key, uploadId, parts);
  res.json({ key });
});

router.post('/multipart/abort', async (req, res) => {
  const { key, uploadId } = req.body;
  await abortMultipartUpload(key, uploadId);
  res.json({ ok: true });
});

export default router;

This is less than 60 lines. It handles small files, large files, cancellation, and type validation. The only infrastructure requirement is an S3 bucket with a lifecycle rule.

Testing the multipart flow locally

You do not need a browser to test this. The AWS CLI can exercise presigned URLs directly.

Generate a test file:

dd if=/dev/urandom of=test-50mb.bin bs=1M count=50

Start your server locally and use the multipart flow from a small Node.js script, or test each stage with curl:

# 1. Initiate
curl -s -X POST http://localhost:3000/api/uploads/multipart/init \
  -H 'Content-Type: application/json' \
  -d '{"filename":"test.bin","contentType":"application/octet-stream","size":52428800}'

# 2. Get URLs for the 5 parts
# 3. PUT each part to its presigned URL
# 4. Complete

For a fully automated test, mock the S3 client. The AWS SDK v3 is designed for this: every command is a plain object, and the client accepts a custom request handler.

import { S3Client } from '@aws-sdk/client-s3';

const mockS3 = new S3Client({
  requestHandler: {
    handle: async (request) => ({
      response: new Response('{}', { status: 200 }),
    }),
  },
});

Unit test your route handlers with a mocked S3 client. Integration test the full flow against LocalStack or a real S3 bucket in a sandbox account.

When this pattern does not fit

Direct-to-S3 upload is not universal. There are legitimate reasons to stream through your server:

  • Real-time processing. If you need to transcode a video, scan for malware, or extract metadata during upload, you may need the bytes to flow through a worker. Consider streaming from S3 to your processor after the direct upload completes instead. It is more steps, but it keeps the upload path fast and resilient.
  • Strict audit requirements. Some compliance regimes require every byte to pass through an audited middlebox. In that case, the architecture is different and you should budget for the infrastructure cost.
  • Tiny files in high volume. The overhead of three API round-trips (init, upload, complete) can exceed the cost of a single PUT through your server for files under 1 KB. Measure for your use case.

For everything else, direct-to-S3 is the default you should start from.

The takeaway

Uploading files through your API server is a tax on every layer of your stack: connections, memory, disk, CPU, timeouts, and your own time debugging why a 3 GB upload fails at 99%. The presigned URL pattern moves the bytes past your infrastructure entirely. Your server handles small JSON requests. S3 handles the storage. The client handles the retry logic per part.

The implementation is 60 lines of server code and 40 lines of client code. The security model is IAM plus short-lived scoped URLs. The failure mode is a single 10 MB part retrying instead of a 2 GB payload starting over. If your product team asks for large file uploads next quarter, this is the implementation that ships on Monday and survives the first traffic spike.

A note from Yojji

Moving large file uploads out of the application tier and into direct cloud storage is the kind of infrastructure decision that looks simple in hindsight but saves enormous operational cost in practice. Yojji’s teams build exactly these kinds of production backend patterns as standard practice, whether they are working with AWS, Azure, or Google Cloud, across full-cycle product builds and dedicated outstaffed engagements.

Yojji is an international custom software development company founded in 2016, with offices in Europe, the US, and the UK. Their engineers specialize in the JavaScript ecosystem (React, Node.js, TypeScript), cloud-native architecture, and the kind of practical infrastructure work that keeps services fast under real load.

Keep Reading