Content Negotiation in Node.js: Accept, Vary, and Serving the Right Representation

Your API endpoint serves an order list. The browser fetches it, renders it fine. Then an internal microservice calls the same endpoint and gets back a 200 with JSON. It parses it, works fine. Then a mobile client on a 3G connection calls it, gets the same JSON payload, and the user waits four seconds for a response that could have been 60% smaller with MessagePack or a compact JSON representation.

You add a new header. The mobile client sends Accept: application/msgpack. Your API checks the header, serializes the response as MessagePack, and the mobile user sees a 60% reduction in transfer size. But now the CDN cache serves the MessagePack variant to the browser, which cannot parse it, and the JSON variant to an internal service that expects Protobuf. You get a ticket: “Intermittent parse failures on order endpoint.”

The root cause is not the serialization format. It is the missing Vary header. Negotiating content without telling the cache about the negotiation is like putting different products in the same warehouse aisle without labeling the shelves. The cache cannot tell which client asked for which variant, so it serves whatever was stored first.

Content negotiation is the HTTP mechanism for serving different representations of the same resource based on what the client can handle and prefers. It is baked into the HTTP specification (RFC 7231, Section 5.3) and it is the difference between a brittle API that only speaks JSON and a flexible one that speaks any format, in any language, compressed or not, without breaking caches between clients.

This post walks through every negotiation header that matters, shows you how to parse quality values correctly, and gives you a production-ready middleware for Express and Fastify that handles JSON, MessagePack, Protobuf, language negotiation, and cache-aware Vary headers. No guesswork, no broken intermediaries.

The three types of negotiation

HTTP defines three dimensions of content negotiation. Each one has its own request header and its own pitfalls.

1. Representation format (what shape the data takes). The client sends Accept with a list of media types and quality weights. The server picks the best match and sets Content-Type on the response.

2. Language (which human language the content should use). The client sends Accept-Language with language tags and quality weights. The server localizes the response body or error messages accordingly.

3. Encoding (how the bytes are compressed on the wire). The client sends Accept-Encoding with compression algorithms and quality weights. The server compresses the body and sets Content-Encoding.

Every one of these requires a corresponding Vary response header. Without Vary, your reverse proxy and CDN will cache responses under the same URL key regardless of negotiation, and clients will get mismatched content.

Why most APIs get Accept parsing wrong

The Accept header looks simple:

Accept: application/json, text/plain;q=0.5, */*;q=0.1

But most server-side implementations do not honor the quality values correctly. The spec says the server must iterate through the client’s list in order of preference (highest q first), find the first media type it can produce, and use that. Wildcards (*/*, application/*) are acceptable matches but score lower than explicit types.

The common mistake is checking Accept.includes('application/json') and calling it done. That ignores quality values entirely. It also breaks when a client sends Accept: application/* and expects any subtype, or when it sends Accept: text/html, application/xhtml+xml, application/xml;q=0.9, */*;q=0.8 (a browser’s default Accept header, which prefers HTML over JSON even though JSON would be faster).

Here is a correct parser in about 20 lines:

interface AcceptEntry {
  type: string;
  subtype: string;
  params: Record<string, string>;
  quality: number;
}

function parseAccept(header: string): AcceptEntry[] {
  const items = header.split(',').map((part) => part.trim());
  return items
    .map((item) => {
      const [mediaRange, ...rawParams] = item.split(';').map((s) => s.trim());
      const [type, subtype] = mediaRange.split('/');
      const params: Record<string, string> = {};
      let quality = 1.0;
      for (const raw of rawParams) {
        const [key, val] = raw.split('=');
        if (key === 'q') {
          quality = parseFloat(val) || 0;
        } else {
          params[key] = val;
        }
      }
      return { type: type || '*', subtype: subtype || '*', params, quality };
    })
    .sort((a, b) => b.quality - a.quality);
}

This gives you a sorted array of the client’s preferences, highest quality first. Now you can match against your server’s capabilities in priority order.

Matching a client Accept list against your formats

Once you have the parsed list, you need to find the best format your server can produce. The matching rules are:

• An explicit match (application/json vs application/json) wins over a wildcard (application/*), regardless of quality (within the same quality tier).
• A wildcard subtype match (application/*) wins over a full wildcard (*/*).
• If no match exists, the server should return 406 Not Acceptable with a list of supported types.

Here is the matcher:

type Format = { type: string; subtype: string };

function negotiateFormat(
  accepted: AcceptEntry[],
  supported: Format[]
): Format | null {
  for (const entry of accepted) {
    for (const candidate of supported) {
      if (
        entry.type === candidate.type &&
        (entry.subtype === candidate.subtype || entry.subtype === '*')
      ) {
        return candidate;
      }
      if (entry.type === '*' && entry.subtype === '*') {
        return supported[0]; // last resort: return the default format
      }
    }
  }
  return null;
}

This handles the common cases correctly. If the client sends Accept: application/json, text/plain;q=0.5 and your server supports JSON, YAML, and plain text, it returns JSON. If the client sends Accept: application/xml and you do not serve XML, it returns null and you respond with 406.

The Vary header: why your cache is poisoning clients

The Vary response header tells intermediaries which request headers were used to select the representation. A CDN that sees Vary: Accept will cache separate entries for each distinct Accept header value. Without it, the CDN caches one response for the URL, and the first client to request the URL determines what every subsequent client gets.

The three most common values are:

Vary: Accept
Vary: Accept-Language
Vary: Accept-Encoding

You can combine them:

Vary: Accept, Accept-Language

But there is a trap: CDNs and browsers have limits on how many Vary variants they cache. If your Accept header is highly variable (every client sends a different order or custom media type), you will see low cache hit rates. The pragmatic fix is to normalize the Accept header into a fixed set of format keys and set Vary based on those keys, not the raw header value.

A better approach for many APIs is to use a format suffix in the URL instead of full content negotiation for the format dimension:

GET /orders/1234.json
GET /orders/1234.msgpack
GET /orders/1234.xml

This bypasses the Vary problem entirely for format negotiation, because each URL is a distinct cache key. You still need Vary for language and encoding negotiation. But for the representation format, a URL suffix or a ?format=json query parameter is simpler and more cache-friendly than full Accept header negotiation.

The trade-off is purity versus practicality. Pure REST says the URL identifies the resource and the Accept header identifies the representation. Practical APIs use suffixes because CDNs handle them better. Choose based on your cache layer, not a textbook.

Language negotiation: Accept-Language in practice

The Accept-Language header follows the same quality-value pattern:

Accept-Language: fr-CH, fr;q=0.9, en;q=0.8, de;q=0.7, *;q=0.5

Parsing it uses the same quality-value splitting logic as Accept. But language negotiation has an additional rule: tag matching is hierarchical. A client that sends Accept-Language: fr-CH (Swiss French) should match fr (any French variant) if fr-CH is not available. The quality value applies to the most specific tag, and fallback matches use the parent tag’s quality.

Here is a practical language negotiator:

function negotiateLanguage(
  header: string | undefined,
  supported: string[],
  defaultLang = 'en'
): string {
  if (!header) return defaultLang;

  const entries = header.split(',').map((part) => part.trim());
  const parsed = entries
    .map((entry) => {
      const [tag, ...params] = entry.split(';').map((s) => s.trim());
      let quality = 1.0;
      for (const p of params) {
        const [k, v] = p.split('=');
        if (k === 'q') quality = parseFloat(v) ?? 1.0;
      }
      return { tag: tag.toLowerCase(), quality };
    })
    .sort((a, b) => b.quality - a.quality);

  for (const pref of parsed) {
    // Exact match
    if (supported.includes(pref.tag)) return pref.tag;
    // Prefix match: fr-CH -> fr
    const base = pref.tag.split('-')[0];
    if (supported.includes(base)) return base;
  }

  return defaultLang;
}

This handles the fr-CH -> fr fallback, ignores wildcard language tags correctly, and returns your default locale when nothing matches.

Accept-Encoding: compression negotiation

Accept-Encoding is the simplest of the three. The client advertises which compression algorithms it supports:

Accept-Encoding: gzip, deflate, br, zstd

The server picks the best algorithm it supports and compresses the response. The key rule from the HTTP spec is: if the client does not send Accept-Encoding, the server MAY still send a compressed response. But if the client sends Accept-Encoding with an empty value or identity;q=0, the server MUST NOT compress.

Most CDNs and browsers handle Accept-Encoding negotiation automatically. Your reverse proxy (Nginx, Envoy) and your CDN (Cloudflare, Fastly, CloudFront) all negotiate and cache by encoding without any application logic. You should generally let them do it. Only implement application-level encoding negotiation if you are building a raw Node.js HTTP server without a proxy.

If you do need it, here is how to check for Brotli support:

function acceptEncoding(header: string | undefined): 'br' | 'gzip' | null {
  if (!header) return null;
  const encodings = header.split(',').map((e) => e.trim());
  if (encodings.includes('br')) return 'br';
  if (encodings.includes('gzip')) return 'gzip';
  return null;
}

Set Vary: Accept-Encoding on the response so CDNs cache different compression variants separately.

A production middleware for Express

Here is a complete middleware that handles format negotiation, language negotiation, and Vary headers, with support for JSON and MessagePack:

import { Request, Response, NextFunction } from 'express';
import { encode } from '@msgpack/msgpack';

interface ContentNegotiationOptions {
  formats: Array<{ type: string; subtype: string; serialize: (data: unknown) => Buffer | string }>;
  languages: string[];
  defaultFormat?: string;
  defaultLanguage?: string;
}

export function contentNegotiationMiddleware(opts: ContentNegotiationOptions) {
  const defaultFormat = opts.defaultFormat || 'json';

  return (req: Request, res: Response, next: NextFunction) => {
    // --- Format negotiation ---
    const acceptHeader = req.headers['accept'] as string | undefined;
    const format = negotiateContentType(acceptHeader, opts.formats, defaultFormat);
    const supportedTypes = opts.formats.map((f) => `${f.type}/${f.subtype}`);

    if (!format) {
      res.status(406).json({
        error: 'Not Acceptable',
        message: `Supported content types: ${supportedTypes.join(', ')}`,
        supportedTypes,
      });
      return;
    }

    // --- Language negotiation ---
    const langHeader = req.headers['accept-language'] as string | undefined;
    const lang = negotiateLanguage(langHeader, opts.languages, opts.defaultLanguage || 'en');

    // --- Store negotiation results on request ---
    (req as any).negotiatedFormat = format;
    (req as any).negotiatedLanguage = lang;

    // --- Override res.json to use negotiated format ---
    const originalJson = res.json.bind(res);
    res.json = function (data: unknown) {
      const serialized = format.serialize(data);
      res.setHeader('Content-Type', format.type + '/' + format.subtype);

      // Set Vary headers
      const varyParts: string[] = [];
      if (acceptHeader) varyParts.push('Accept');
      if (langHeader) varyParts.push('Accept-Language');
      if (varyParts.length > 0) {
        res.setHeader('Vary', varyParts.join(', '));
      }

      return res.send(serialized);
    };

    next();
  };
}

function negotiateContentType(
  acceptHeader: string | undefined,
  formats: ContentNegotiationOptions['formats'],
  defaultFormat: string
): ContentNegotiationOptions['formats'][0] | null {
  if (!acceptHeader) {
    return formats.find((f) => f.subtype === defaultFormat) || formats[0];
  }

  const parsed = parseAccept(acceptHeader);
  for (const entry of parsed) {
    for (const fmt of formats) {
      if (entry.type === fmt.type && (entry.subtype === fmt.subtype || entry.subtype === '*')) {
        return fmt;
      }
      if (entry.type === '*' && entry.subtype === '*') {
        return formats.find((f) => f.subtype === defaultFormat) || formats[0];
      }
    }
  }
  return null;
}

Register it in your app:

import express from 'express';
import { contentNegotiationMiddleware } from './negotiation';

const app = express();

app.use(contentNegotiationMiddleware({
  formats: [
    {
      type: 'application', subtype: 'json',
      serialize: (data) => JSON.stringify(data),
    },
    {
      type: 'application', subtype: 'msgpack',
      serialize: (data) => encode(data as any),
    },
  ],
  languages: ['en', 'fr', 'de', 'ja'],
  defaultLanguage: 'en',
}));

app.get('/orders', (req, res) => {
  const orders = [{ id: 1, total: 29.99 }];
  res.json(orders); // Uses negotiated format
});

Test it:

# JSON (default)
curl -H 'Accept: application/json' http://localhost:3000/orders

# MessagePack (check with xxd or hexdump)
curl -H 'Accept: application/msgpack' http://localhost:3000/orders | xxd

# 406 for unsupported format
curl -H 'Accept: application/xml' http://localhost:3000/orders

# Language negotiation
curl -H 'Accept-Language: fr' http://localhost:3000/orders

Content negotiation with Fastify

Fastify has built-in content type serialization that makes this cleaner:

import Fastify from 'fastify';

const app = Fastify();

// Register a JSON serializer (built-in)
app.addContentTypeParser('application/json', { parseAs: 'string' }, (req, body, done) => {
  done(null, body);
});

// Add a MessagePack serializer
import { encode } from '@msgpack/msgpack';

app.addSchema({
  $id: 'order',
  type: 'object',
  properties: {
    id: { type: 'integer' },
    total: { type: 'number' },
  },
});

app.get('/orders', {
  schema: {
    response: {
      200: {
        content: {
          'application/json': { schema: { type: 'array', items: { $ref: 'order#' } } },
          'application/msgpack': { schema: { type: 'array', items: { $ref: 'order#' } } },
        },
      },
    },
  },
  preHandler: (req, reply, done) => {
    reply.header('Vary', 'Accept');
    done();
  },
}, async (req, reply) => {
  const orders = [{ id: 1, total: 29.99 }];
  const accept = req.headers.accept || '';

  if (accept.includes('msgpack')) {
    return reply.type('application/msgpack').send(encode(orders));
  }

  return orders;
});

Fastify’s schema-driven serialization is faster than runtime checks because it pre-compiles serializers per content type. The Vary header must be set on every response to prevent cache collisions.

Common pitfalls and how to avoid them

Pitfall #1: Vary starvation. Setting Vary: Accept on a high-traffic endpoint where every client sends a different Accept header (some include */*;q=0.8, some do not) blows up your cache key space. Each unique Accept string becomes a separate cache entry. The fix is to normalize: strip quality values and wildcards after negotiation, then set a fixed set of Vary entries for the formats you actually produce.

Pitfall #2: The browser default Accept header. Browsers ship with Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8. If your API endpoint returns JSON, the browser’s Accept header prefers HTML and XML. Your negotiation middleware must match application/json correctly against the wildcards and quality values, not just check for includes('json').

Pitfall #3: Assuming Accept-Encoding is always present. Some HTTP clients (especially in IoT and embedded environments) omit Accept-Encoding entirely. The spec permits server-side compression in this case, but many CDNs will not compress responses without the header. If you need compression for clients that do not advertise it, set a conservative fallback like gzip level 1.

Pitfall #4: Content-Type on error responses. When negotiation fails and you return 406 Not Acceptable, set Content-Type: application/json (or your default format) explicitly. Without it, some clients will fail to parse the error body because they were expecting a different format.

The practical takeaway

Content negotiation is one of those HTTP features that looks unnecessary until you need it, and by then your cache is already poisoning clients and your error responses are unparseable.

Here is the checklist:

1. Parse Accept correctly with quality values. Do not use includes(). Use a proper parser that sorts by q-value and matches wildcards.
2. Set Vary on every negotiated response. Every unique negotiation dimension (Accept, Accept-Language, Accept-Encoding) needs a corresponding Vary entry. Your CDN cannot cache correctly without it.
3. Normalize Vary to reduce cache fragmentation. Strip quality values after negotiation and collapse the Vary space into your supported format list.
4. Consider URL suffixes for format negotiation. GET /orders/1234.json is simpler than GET /orders/1234 with Accept: application/json, and it avoids Vary fragmentation entirely. Use Accept for the dimensions that truly vary per client (language, maybe encoding).
5. Return 406 with a list of supported types when negotiation fails. Raw 400 Bad Request or 415 Unsupported Media Type gives the client no actionable information. A 406 with a clear body tells them exactly what formats you support.
6. Always test with curl. Send different Accept headers, check the Content-Type and Vary on the response, and verify the CDN caches each variant independently. Automate this in your CI pipeline with a test that asserts the right headers.

The APIs that handle negotiation best are the ones that treat it as a first-class concern, not an afterthought. They define their supported formats, languages, and encodings at the router level, apply Vary headers automatically, and test that the cache layer respects them. Your future self, debugging a production incident at 2 AM, will thank you for the extra header.

A note from Yojji

HTTP content negotiation seems like a textbook spec detail until your CDN serves the wrong format to thousands of concurrent clients. Getting the Accept, Vary, and Content-Type dance right requires more than reading RFC 7231. It requires understanding how every intermediary in your stack handles cache keys, and that changes from Cloudflare to Fastly to a custom Varnish setup. Yojji is an international custom software development company founded in 2016, with offices in Europe, the US, and the UK. Their teams build and tune API infrastructure for the JavaScript ecosystem (React, Node.js, TypeScript) on AWS, Azure, and Google Cloud, handling the full cycle from architecture to production monitoring so your API speaks every format your clients need without breaking caches along the way.