413 Request Entity Too Large

Before you start

• A way to repeat the failing request, such as curl with a file of the same size.
• Read access to the nginx configuration and error log on the server, or kubectl access to the Ingress and the ingress-nginx ConfigMap.
• Permission to reload nginx or edit the Ingress, since the fix is a configuration change.

Work out which cause you have

A few questions to narrow the list. Every answer ends in a command that confirms or rules the cause out — this cannot see your cluster, so nothing here is a certainty until you have checked.

Check it with a tool

• Migrating off Ingress NGINX→

Or diagnose it manually

In this order. The first command usually contains the whole answer.

1. Step 1

   curl -sS -X POST --data-binary @body.bin -w '\nstatus=%{http_code} sent=%{size_upload}\n' https://<host>/<path>

   Repeats the failing request with a body of the same size, and shows who answered and how much was sent first.

   Look for nginx's HTML page, the application's own format, or a CDN's page; a small sent value means the limit was enforced from Content-Length

2. Step 2

   sudo grep 'too large body' /var/log/nginx/error.log | tail -5

   nginx logs every body it rejects for size, with the size.

   Look for client intended to send too large body: <bytes> bytes

3. Step 3

   sudo nginx -T 2>/dev/null | grep -n -B3 client_max_body_size

   Prints the full loaded configuration, so you can see which block sets the limit for your path.

   Look for The value in the block that serves the path, or no setting, meaning 1m

Every cause, and how to fix it

Ordered by how often each one turns out to be the answer.

sudo nginx -T 2>/dev/null | grep -n client_max_body_size
sudo grep 'too large body' /var/log/nginx/error.log | tail -5

# in the server or location block that serves the upload path:
#   client_max_body_size 20m;
sudo nginx -t && sudo systemctl reload nginx

kubectl get ingress <name> -n <namespace> -o jsonpath='{.metadata.annotations}'

kubectl annotate ingress <name> -n <namespace> nginx.ingress.kubernetes.io/proxy-body-size=20m --overwrite

curl -sS -X POST --data-binary @body.bin -o - http://127.0.0.1:<app-port>/<path>

curl -sS -D - -o /dev/null -X POST --data-binary @body.bin https://<host>/<path> | grep -iE '^(HTTP|server|via|cf-ray)'

How to know it is actually fixed

• ✓The original request, or a curl with a body of the same size, returns the application's normal response instead of 413.
• ✓A body above the new limit still gets 413, which shows the limit is set where you meant it to be rather than switched off.
• ✓No new client intended to send too large body lines appear in the error log for legitimate uploads.

Stopping it happening again

• Decide the largest upload each endpoint accepts, and set the same number in nginx, the ingress and the application, documented in one place.
• Make the upload form check the file size in the browser first and show a clear message, since browsers do not display nginx's 413 page well.
• For files much larger than a few megabytes, upload directly to object storage with pre-signed URLs instead of through your web server.

Security considerations

• The limit is a protection, not an obstacle. It bounds how much an anonymous client can make nginx buffer, so raise it only in the location that receives uploads and only to the size you need, rather than setting 0 or a large value server-wide.
• Raising nginx's limit moves the load to the application. Make sure the upload handler streams to disk or object storage and enforces its own maximum, instead of reading the whole body into memory.
• Keep authentication and rate limits in front of any location you open up for large bodies, so the larger allowance is only available to people allowed to upload.