← Back to Kubernetes Resource Limits Check

Why a missing resource limit can take down an entire node

Memory: two different mechanisms, depending on scope

A container with no memory limit is free to grow. Kubernetes itself does not stop it -- the only thing that eventually does is the Linux kernel's out-of-memory (OOM) killer, once the node runs genuinely short on memory. The OOM killer's scoring (oom_score_adj, weighted heavily by a cgroup's memory limit relative to what it is using) is not guaranteed to pick the container that grew; a process elsewhere on the node with a worse score can be the one killed instead.

Below that hard limit, Kubernetes' own kubelet runs node-pressure eviction: when a node crosses a memory (or disk) threshold, the kubelet evicts whole pods to relieve it, ranked first by QoS class (BestEffort evicted before Burstable, Burstable before Guaranteed) and then by how far each pod's actual usage exceeds its own requests. A container with no memory request at all is treated as requesting nothing, which makes any usage look maximally over-request -- ranking it for eviction first, regardless of how little it is actually using at that moment.

Ephemeral storage: the one most setups still miss

A container's logs, temporary files, and its own writable layer all draw on the node's local (ephemeral) storage, and Kubernetes tracks that the same way it tracks memory: as a resource a container can request and limit, documented under Resource Management for Pods and Containers. With no limit set, a container that logs too verbosely or leaves temp files behind can fill the node's disk. Once the node crosses its own disk-pressure threshold, the kubelet evicts pods to reclaim space -- again at the node level, not scoped to the container that filled it. Most teams reach for CPU and memory limits as a matter of habit and never add this one, because nothing prompts them to until a node has already gone into disk pressure.

CPU limits: a real trade-off, not a missing safety rail

CPU is different from memory and disk in one important way: it is compressible. A container over its CPU request is not killed, it is merely scheduled less. But a CPU limit, once set, is enforced by the kernel's CFS bandwidth controller, which grants a fixed quota once per scheduling period (by default 100 milliseconds) with no rollover of unused quota into the next period. A container with bursty usage -- busy for a slice of a period, idle for the rest, repeated -- can exhaust its quota partway through a period and be throttled for the remainder, even while the node as a whole has CPU sitting idle. This is tracked upstream as kubernetes/kubernetes#67577 and has led several engineering teams to publish their own "why we stopped setting CPU limits" write-ups after tracing latency spikes back to exactly this. It is not a reason to skip CPU limits everywhere -- strict multi-tenant isolation is a real, legitimate reason to keep them -- which is why this tool reports a set CPU limit as informational rather than as a problem: the point is to make the trade-off visible, not to make the decision for you.

The one this tool also catches: a request above its own limit

Kubernetes' API server validates, at admission time, that a container's resource request for a given type is never greater than its limit for that same type. A manifest that violates this is not deployed with a warning -- kubectl apply or a rolling helm upgrade is rejected outright. This is usually a typo (a limit lowered during a tuning pass without lowering the request to match, or the reverse) rather than a deliberate choice, and catching it before the apply step saves the round trip of a failed deploy.

What this tool looks for, and where it stops

It reads a manifest's YAML directly -- resources.requests and resources.limits on every container and init container across every Pod, Deployment, StatefulSet, DaemonSet, ReplicaSet, Job, and CronJob it finds -- and reports what is present, what is missing, and whether a request exceeds its own limit. It has no kubeconfig and makes no API server call, which is what makes it safe to run in CI against a manifest repository with no cluster credentials granted at all.

What it cannot do is tell you the right number for a request or limit -- that depends on what your workload actually uses, which only your own metrics (kubectl top, or whatever monitoring stack you run) can answer. It checks that a number is present and internally consistent, not that it is the correct one.

Common questions

What actually kills a container with no memory limit?

Two different mechanisms, depending on scope. If the container alone grows past what its node has free, the Linux kernel's out-of-memory killer picks a process to kill based on its own scoring -- not necessarily the container that grew, if another process on the node has a worse score. If the whole node is short on memory across many containers, the kubelet's own node-pressure eviction acts first, evicting whole pods ranked by QoS class and how far each is over its own requests -- a container with no request at all is ranked worst regardless of what it is actually using.

Why does an ephemeral-storage limit matter when memory and CPU are usually the ones people set?

Because a container's logs, temp files, and writable container layer all count against the node's local disk, and Kubernetes tracks that the same way it tracks memory: as a resource a container can request and limit. Skip the limit and a container that logs too verbosely, or writes temp files it never cleans up, can fill the node's disk. Once the node crosses its disk-pressure threshold the kubelet starts evicting pods to reclaim space -- not only the one that filled it, because eviction acts at the node level.

Should I always set a CPU limit along with the request?

Not automatically. A CPU limit is enforced through the kernel's CFS bandwidth controller, which grants a fixed quota once per scheduling period (typically 100ms) with no rollover of unused quota into the next period. A container with bursty CPU use -- busy for 20ms, idle for the rest of a period, repeated -- can be throttled inside that pattern even while the node as a whole has spare CPU sitting idle. This is a well-documented behaviour, tracked upstream as kubernetes/kubernetes#67577, and it is why several engineering teams have published posts about removing CPU limits while keeping CPU requests. A limit is still the right call for strict multi-tenant isolation; the point is that it is a trade-off to make deliberately, not a box to tick alongside every other resource field.