The Practical Developer

Kubernetes Resource Requests And Limits: The Numbers That Decide If Your Cluster Is Stable

October 11, 2024 by The Practical Developer

Most teams set CPU and memory requests by guessing. The result is over-provisioning that wastes money or under-provisioning that causes evictions. Here is the practical method for picking each number, the difference between requests and limits, and why CPU limits are often a mistake.

kubernetes

devops

performance

A control panel — the right metaphor for the dials of pod resource configuration

The team’s pods have cpu: 500m and memory: 512Mi requests. Nobody remembers why. Some pods get OOMKilled at 3 a.m. Others use 5% of their request. The cluster is over-provisioned by 60% and still has reliability incidents. The on-call asks “what should the resources actually be?” and nobody has an answer.

Setting resource requests and limits is one of those Kubernetes details that seems trivial and is actually load-bearing. Wrong numbers mean either burning money or being on call for OOM kills. The right numbers come from measurement, not guesses, and the methodology takes an afternoon.

This post is the working method: how to pick CPU and memory requests, the difference between requests and limits, and why CPU limits are often a mistake.

Requests vs limits in 30 seconds

Requests are what the scheduler reserves. A pod with cpu: 500m request will only be placed on a node with 500 millicores of free CPU capacity. The pod is guaranteed this much.

Limits are the cap. A pod with cpu: 1 limit cannot use more than 1 core, even if the node has spare CPU.

Behavior:

  • CPU: When over the limit, the kernel throttles the process. CPU usage flatlines at the limit; the process slows down but doesn’t die.
  • Memory: When over the limit, the kernel OOM-kills the process. The pod restarts.

These are very different failure modes. CPU throttling is recoverable; memory OOM is not.

How to pick memory request and limit

Memory is easier because it’s deterministic — your process uses approximately the same memory for the same workload over time.

Step 1: Measure. Run the service in production (or load-tested staging) for a week. Look at memory usage. Take the p99 over the period.

quantile_over_time(0.99, container_memory_usage_bytes{pod=~"api-.*"}[7d])

Add a 30% safety margin. This is your memory limit.

Step 2: Pick the request. A common pattern: request = limit. This makes the pod “Guaranteed” QoS, which is the most stable behavior. Another pattern: request = 0.7 * limit, which lets the scheduler pack pods more densely (Burstable QoS).

For most production services, request = limit is the right default. The safety from a Guaranteed pod (won’t be evicted under memory pressure) outweighs the packing efficiency.

Memory configuration:

resources:
  requests:
    memory: 768Mi
  limits:
    memory: 768Mi

If the pod actually uses 600 MB at p99, the 168 MB of headroom absorbs spikes.

How to pick CPU request

CPU is trickier because it’s bursty. Your process might use 2 cores for 100ms, then 0.1 core for 9 seconds.

Step 1: Measure CPU usage. The metric you want is steady-state utilization, not peak. P95 over 1-minute windows is usually a good signal.

quantile_over_time(0.95, rate(container_cpu_usage_seconds_total{pod=~"api-.*"}[1m])[7d:])

Step 2: Set the request to ~p95 of steady-state. This is what the pod “deserves” most of the time. Add a small safety margin (10-20%).

For a service that sits at 200m most of the time and bursts to 800m occasionally:

resources:
  requests:
    cpu: 250m
    # No CPU limit — see below.

Why CPU limits are often a mistake

This is the controversial one: in many real production setups, do not set CPU limits.

The reasoning:

  • CPU is shareable. Unlike memory, CPU not used by one pod is available to others. There’s no “leak” to prevent.
  • Throttling hurts performance unnecessarily. A request that needed 800m for 50ms gets throttled and takes 200ms instead. Latency increases, autoscaling fires, total cluster cost goes up.
  • CFS quota is unfair. The kernel CPU scheduler (CFS) has documented bugs that throttle pods more aggressively than they should be.

The opposing view: limits prevent runaway processes from starving the rest. Valid for pods that may legitimately consume infinite CPU (poorly-behaved background jobs).

For most application pods (HTTP services, queue workers with bounded work):

  • Set CPU request to the steady-state p95.
  • Don’t set CPU limit.
  • Set memory request and limit to the same value.

This is the production setup that gives you the best behavior on shared nodes.

QoS classes

Kubernetes assigns each pod a Quality of Service class based on its resource config:

  • Guaranteed: requests == limits for all containers. Most stable. Last to be evicted under pressure.
  • Burstable: requests < limits, or only some resources have requests. Default behavior; reasonable for most pods.
  • BestEffort: no requests, no limits. First to be evicted. Use only for non-critical batch work.

For production HTTP services and important workers, target Guaranteed for memory (request == limit) and Burstable for CPU (request set, no limit).

Probes and the LimitRanger

Two pieces that interact with resource config:

LimitRanger is a built-in admission controller that sets defaults if you forget:

apiVersion: v1
kind: LimitRange
metadata:
  name: default
spec:
  limits:
  - default:
      memory: 512Mi
      cpu: 200m
    defaultRequest:
      memory: 256Mi
      cpu: 100m
    type: Container

Useful for catching pods that ship without resource specs at all. Not a substitute for tuning per service.

Vertical Pod Autoscaler (VPA) in recommendation mode (updateMode: Off) watches pod usage and suggests right-sized requests:

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: api
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  updatePolicy:
    updateMode: "Off"

After a few days of running, kubectl describe vpa api shows recommended values. Apply at deploy time.

A working configuration

For a Node.js API that benchmarks at 250m CPU, 600 MB memory at p99 under typical load:

spec:
  containers:
  - name: api
    image: ghcr.io/example/api:abc123
    resources:
      requests:
        cpu: 300m
        memory: 800Mi
      limits:
        memory: 800Mi
        # Intentionally no CPU limit.

The 300m CPU is “what we deserve,” with no cap (lets us burst up if a node has spare). The 800Mi memory is “what we actually use plus headroom,” capped to prevent runaway leaks.

OOM-killed pods: how to debug

When a pod is OOM-killed:

kubectl describe pod api-abc123
# Look for: "Last State: Terminated", "Reason: OOMKilled"
kubectl logs api-abc123 --previous
# Logs from the killed container.

Common causes:

  1. Memory limit too low. Bump it. Re-measure.
  2. Memory leak. Heap grows unboundedly. Profile and fix.
  3. Cache or connection pool sized wrong. A connection pool of 100 connections at 5 MB each is 500 MB, which may exceed the limit.
  4. Sudden traffic spike. Per-request memory * concurrent requests > limit.

Don’t just bump the limit — investigate why the limit was hit. Often the right fix is “reduce the cache size” or “fix the leak,” not “give it more memory.”

Cluster-wide views

A few queries that surface cluster-wide issues:

# Pods using > 80% of their memory limit
container_memory_usage_bytes / on(pod, container) container_spec_memory_limit_bytes > 0.8

# Pods being CPU-throttled
rate(container_cpu_cfs_throttled_seconds_total[5m]) > 0

# Cluster-wide CPU request over-allocation
sum(kube_pod_container_resource_requests{resource="cpu"}) / sum(kube_node_status_allocatable{resource="cpu"})

The third one is the “are we over-provisioned” metric. If your cluster is at 30% CPU request allocation, you’re paying for 70% of capacity that nobody is using. Right-sizing requests pays for itself fast.

Common mistakes

1. Copying resource specs from another service. Each service has its own profile. Measure each.

2. Using percentile over short windows. A 5-minute p95 misses the once-an-hour spike. Use 7-day windows or longer.

3. Setting CPU limit equal to request “to be safe.” Now you’re throttled at the request level, which is exactly what you don’t want. Either no limit, or limit much higher than request.

4. Ignoring sidecar containers. Service mesh sidecars (Envoy, Linkerd) consume real CPU and memory. Add their requirements to the pod total.

The takeaway

CPU and memory requests are not aspirational numbers. They are the contract between your pod and the scheduler. Measure first; don’t guess. Memory: request == limit, sized to p99 + 30%. CPU: request to steady-state p95; no limit (in most cases). Use VPA in recommendation mode for ongoing tuning.

The team that does this well has a stable cluster at 60-70% utilization. The team that doesn’t has a 30%-utilized cluster with regular OOM incidents. The difference is one afternoon of measurement.

A note from Yojji

The kind of platform-engineering judgment that turns “we set memory to 512Mi because that was the default” into “we measured and set it to what the workload actually needs” is the kind of long-haul DevOps work Yojji’s teams build into the Kubernetes platforms they ship for clients.

Yojji is an international custom software development company founded in 2016, with teams across Europe, the US, and the UK. They specialize in the JavaScript ecosystem, cloud platforms (AWS, Azure, GCP), and Kubernetes operations — including the resource-tuning work that decides whether your cluster is stable and cost-efficient or wasteful and incident-prone.