Kubernetes · what happens if

What happens if a container exceeds its memory limit?

Short answer

Once the kernel cannot reclaim enough memory to stay under the limit, it kills processes in the container, and the container ends when its main process dies. There is no warning, no signal your application can catch, and no chance to shut down cleanly: limits.memory is a hard limit, enforced by the OOM killer rather than by slowing the process down.

First published 4 Sept 2026

On this page · 8 sections

Starting state

A container with limits.memory: 512Mi, running normally, whose working set is growing — a leak, a large request, or an unbounded cache.

Words used on this page

memory limit: resources.limits.memory: the most memory a container may use. The container runtime turns it into a cgroup limit that the kernel enforces.
memory request: resources.requests.memory: the memory the scheduler reserves for the container on a node. It is not a ceiling.
cgroup: Control group: the Linux kernel feature that groups a container's processes so their memory and CPU use can be limited and counted.
reclaim: The kernel freeing memory it can get back, such as file cache, so that an allocation can succeed without going over the limit.
OOM killer: The part of the Linux kernel that, when memory runs out and reclaim cannot help, ends processes with SIGKILL to free memory.
OOMKilled: The termination reason the container runtime reports for a container ended by the cgroup OOM killer. It appears in kubectl get pods and kubectl describe pod.
exit code 137: 128 plus 9, the number of SIGKILL: how a container killed by that signal is reported.
CrashLoopBackOff: What kubectl shows while the kubelet waits before restarting a container that keeps failing. The wait doubles after each failure, up to a cap.
node-pressure eviction: The kubelet ending whole Pods because the node itself is short of memory. It picks Pods by their usage over requests and their priority, not by their limits.

The sequence

Colour marks which component acts. Steps sharing a timestamp happen concurrently — nothing here waits for anything else unless it says so.

your app
kernel
kubelet

before anything visibleyour app
Your process allocates memory and does not release it — a cache with no ceiling, a query loading an unbounded result set, or a runtime whose heap is sized against the node rather than the container. Nothing is wrong yet.
What you would see Normal operation. This is the phase where the problem is cheap to find and nobody is looking.
as usage growskernel
The cgroup's memory usage approaches its limit. The kernel starts reclaiming what it can: page cache, buffers, anything droppable.
What you would see Nothing in your application logs. If you are collecting container metrics you would see usage flatten near the ceiling while performance degrades.
at the limitkernel
Reclaim can no longer satisfy the next allocation. The OOM killer is invoked in the container's cgroup and sends SIGKILL. On cgroup v2 nodes, Kubernetes 1.28 and later set memory.oom.group, so every process in the container is killed together. On cgroup v1, or with the kubelet's singleProcessOOMKill set, it kills one process, and the container survives unless that process is PID 1.
What you would see An OOM entry in the node's kernel log. Your application's last log line is whatever it happened to be doing — there is no shutdown message, because SIGKILL cannot be handled.
immediately afterkubelet
The container runtime reports that the container exited with code 137 and, as the CRI requires for a cgroup OOM kill, the reason OOMKilled. The kubelet records both in the Pod's status.
What you would see kubectl describe pod shows Last State: Terminated, Reason: OOMKilled, Exit Code: 137.
per restart policykubelet
With the default restartPolicy: Always, the container is restarted. Its memory usage begins from zero.
after restartyour app
Your process starts again with an empty heap and begins the same growth. If the cause is a leak rather than an under-sized limit, the sequence from here is identical — which is why the interval between kills is the useful signal.
if it recurskubelet
Repeated kills produce an exponentially increasing restart delay, and the pod's status becomes CrashLoopBackOff — which is a symptom of the OOM, not a separate problem.
What you would see Restart count climbing, with the gaps between restarts getting longer: by default 10, 20, 40 seconds and so on, capped at 5 minutes, and reset once the container runs for 10 minutes without a problem. Since Kubernetes 1.35 a node's kubelet can be configured with a lower cap.

What you actually control

Separated from what happens regardless, because conflating the two is how people end up tuning the wrong thing.

limits.memory: The kill line. Set it above your observed peak with headroom. Raising it is the right fix for a container that was simply under-provisioned, and the wrong fix for a leak — it buys time and nothing else.
requests.memory: Does not affect this kill: the limit alone sets it. It affects scheduling, and what happens under node pressure: the kubelet evicts first the pods whose usage exceeds their requests, ordered by priority and then by how far over they are, and if the node runs out of memory before that, the kernel's OOM killer favours containers using a lot relative to their requests. That is a different failure with a different signature.
Application-level bounds: The real fix for causes like these: a cache with no maximum size, a query that loads an unbounded result set, a runtime heap larger than the container limit.
The runtime's own heap setting: Worth checking explicitly. Current JVMs read the container's limit by default, but an old one, or one started with container support turned off, sizes its heap against the node's memory and is killed long before it thinks it is under pressure. For Node.js, set --max-old-space-size below the limit rather than trusting the default.

Try it yourself

Before you start

A Kubernetes cluster you can experiment on, not production, and kubectl configured for it. The example uses its own namespace, created with kubectl create namespace mem-example.
Permission to create a namespace and Pods in it, and to read Pod status and events.
Network access for the cluster to pull polinux/stress from Docker Hub, the image the example uses to allocate memory.
Your nodes' cgroup version, because it decides whether the kernel kills the whole container or one process in it. On a node, stat -fc %T /sys/fs/cgroup/ prints cgroup2fs for cgroup v2 and tmpfs for v1; Kubernetes deprecated cgroup v1 in 1.35.
Optionally, shell access to a node to read the kernel's own record of the kill in its log. Managed clusters often do not give you that access.

Watching a container killed for exceeding 100 MiB

Quoted from the Kubernetes task page Assign Memory Resources to Containers and Pods, section Exceed a Container's memory limit; not reproduced here, because no cluster was available. The Pod memory-demo-2 has a 50 MiB memory request and a 100 MiB limit, and runs stress --vm 1 --vm-bytes 250M --vm-hang 1, which tries to allocate 250 MiB. The YAML is the page's excerpt of the container status; the ... at the start of the event lines is the page's own. The output dates from 2017, so your field order and event wording may differ.

Command

kubectl get pod memory-demo-2 --namespace=mem-example
kubectl get pod memory-demo-2 --output=yaml --namespace=mem-example
kubectl get pod memory-demo-2 --namespace=mem-example
kubectl describe pod memory-demo-2 --namespace=mem-example

Output · as shown in Kubernetes — Assign Memory Resources to Containers and Pods: Exceed a Container's memory limit

$ kubectl get pod memory-demo-2 --namespace=mem-example
NAME            READY     STATUS      RESTARTS   AGE
memory-demo-2   0/1       OOMKilled   1          24s

$ kubectl get pod memory-demo-2 --output=yaml --namespace=mem-example
lastState:
   terminated:
     containerID: 65183c1877aaec2e8427bc95609cc52677a454b56fcb24340dbd22917c23b10f
     exitCode: 137
     finishedAt: 2017-06-20T20:52:19Z
     reason: OOMKilled
     startedAt: null

$ kubectl get pod memory-demo-2 --namespace=mem-example
NAME            READY     STATUS    RESTARTS   AGE
memory-demo-2   1/1       Running   2          40s

$ kubectl describe pod memory-demo-2 --namespace=mem-example
... Normal  Created   Created container with id 66a3a20aa7980e61be4922780bf9d24d1a1d8b7395c09861225b0eba1b1f8511
... Warning BackOff   Back-off restarting failed container

How to read it

OOMKilled in the STATUS column, with RESTARTS at 1: the container tried to go past its 100 MiB limit, was killed, and has already been started again.
In the container status, exitCode: 137 is 128 plus 9, the number of SIGKILL, a signal no process can catch. reason: OOMKilled comes from the container runtime, which the CRI requires to report that reason for a container ended by the cgroup OOM killer; the kubelet records it in the Pod's status.
Sixteen seconds later the container is Running with RESTARTS at 2. Nothing about it changed, so the same allocation will be killed again: the page describes the container being killed and restarted over and over.
Back-off restarting failed container is the kubelet waiting before the next restart. By default the waits are 10, 20, 40 seconds and so on, up to 5 minutes; while one is in effect, kubectl get pods shows CrashLoopBackOff.

Where this goes wrong

No limit is set at all
Unless the namespace gives it a default limit through a LimitRange, the container can grow until the node is short of memory. Then the kubelet evicts pods, or, if memory runs out before it reacts, the kernel's OOM killer picks containers across the node; either way the victims can be other people's. A memory problem in one workload becomes an incident in an unrelated one.
The limit is raised without finding the cause
The interval between kills gets longer, which reads as a fix. A leak reaches any ceiling eventually, so this converts a frequent, obvious failure into a rare, confusing one.
The kill lands on a worker mid-job
SIGKILL means no cleanup, so any job that was not idempotent and re-queued is simply lost. This is the case where the silence matters most.

If it goes wrong

kubectl get pods cycles through OOMKilled, Running and CrashLoopBackOff for the same Pod: That is a container being killed at its limit and restarted with back-off. Compare its real peak with the limit. If it was simply under-provisioned, raise the limit; if usage grows until each kill, find the leak first.
Pods fail with the reason Evicted instead of OOMKilled: That is node-pressure eviction, not the container's own limit: the node ran short of memory. Check the node's MemoryPressure condition with kubectl describe node, and set requests close to real usage, because Pods using more than they requested are evicted first.
A process inside the container dies, but the container keeps running and is never marked OOMKilled: On cgroup v1, or with the kubelet's singleProcessOOMKill set, the OOM killer ends one process, and the container lives on unless that was PID 1. On a cgroup v2 node, the oom_kill count in the container cgroup's memory.events file shows it. Treat it like any other OOM kill: bound the memory or raise the limit.
A Java application is OOMKilled although -Xmx is below the container's limit: -Xmx bounds only the heap, while the cgroup counts all of the process's memory. Leave headroom between the heap size and the limit, or size the heap as a share of the limit with -XX:MaxRAMPercentage, which current JVMs apply to the container's limit.
OOMKilled although the process's own memory looks small: Look for an emptyDir with medium: Memory. Files in it are tmpfs pages that count against the container's memory limit; give the volume a sizeLimit or move the data to disk.

The part worth remembering

The asymmetry between CPU and memory is the thing to internalise. Exceeding a CPU limit throttles you — unpleasant, survivable, and visible as latency. Exceeding a memory limit kills you, because there is no way to give a process less memory than it just asked for.

That is why the advice on the two differs so much, and why a memory limit deserves careful sizing while a CPU limit is a trade-off: without one a container can use any spare CPU on the node, and with one the kernel holds it back at the ceiling even when the node is idle.

It also explains why OOM kills are so quiet. Every other failure mode gives your application a chance to say something. This one does not, so the evidence is all outside your application: the container's termination reason, the node's kernel log, and the oom_kill count in the container cgroup's memory.events file.

Security considerations

A container without a memory limit is a risk to its neighbours. It can use all of the node's memory, and the eviction or kernel OOM kill that follows can land on other teams' Pods. Give every namespace a default limit with a LimitRange, so a Pod that forgets one still gets one.
Anything that writes into a memory-backed emptyDir spends the container's memory limit; without a sizeLimit, the volume can grow to that limit. If users can make your application write there, for example through uploads, they can get it OOM-killed on demand.
An OOM kill can land mid-write, with no chance to clean up. Write files under a temporary name and rename them into place, and make jobs idempotent, so a half-finished write is detectable and safe to repeat.
The example pulls polinux/stress, an image published by a Docker Hub account rather than by the Kubernetes project. Run it only in a throwaway namespace, and do not copy it into real workloads.

Sources

The behaviour this page describes, as documented by the projects themselves.

Did this get you to an answer?

No text box on purpose — please do not paste production logs anywhere

Something wrong or out of date? Report it