Nginx as a Reverse Proxy: The Config, and What Breaks

Getting nginx to proxy to your application takes about six lines. Getting it to behave correctly when the application is slow, restarting, or briefly gone takes rather more, and that second part is what people actually end up debugging.

So this covers the config, and then the three failures it produces — because a reverse proxy's whole job is standing between a client and something that might not answer, and how it handles "might not answer" is the interesting part.

Who this is for

You have nginx proxying to your application and you are now debugging 502s. You do not need to tune anything exotic: four headers, and the difference between 502 and 504, cover most of it.

The minimum that works

server {
    listen 80;
    server_name example.com;

    location / {
        proxy_pass http://127.0.0.1:3000;

        # Without these your application sees nginx as the client: every
        # request appears to come from 127.0.0.1 on http, which breaks
        # redirect generation, rate limiting and any IP logging you rely on.
        proxy_set_header Host              $host;
        proxy_set_header X-Real-IP         $remote_addr;
        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

The four headers are the part most short examples omit and most applications need. If your app generates http:// links behind an HTTPS proxy, or logs every request as coming from localhost, this is why.

For WebSockets, add the upgrade headers to that location — without them the handshake fails and the connection falls back or drops:

        proxy_http_version 1.1;
        proxy_set_header Upgrade    $http_upgrade;
        proxy_set_header Connection "upgrade";

The three failures

502 Bad Gateway — nothing usable came back

nginx reached for your application and got an answer it could not use, or no answer at all: connection refused, connection reset, the process died mid-response.

The important property: your application logs may be completely silent, and that silence is evidence. If nothing arrived, the fault is between nginx and the upstream — the process is down, bound to a different port or interface, or being refused by something in between.

# What nginx itself saw. This is the log that matters and the one people skip.
sudo tail -50 /var/log/nginx/error.log

# Is anything listening where proxy_pass points?
ss -ltnp | grep 3000

A detail that catches people: proxy_pass http://localhost:3000 can resolve to both ::1 and 127.0.0.1, and nginx round-robins between them. If your application binds IPv4 only, every attempt on ::1 is refused and logged; by default nginx retries on 127.0.0.1, so you get error-log noise and added latency — and a 502 when localhost resolves only to ::1 or retries are off — all while curl localhost:3000 works from your shell. Use 127.0.0.1 explicitly.

The full cause list is on the 502 Bad Gateway page.

504 Gateway Timeout — it did not answer in time

Different failure, different fix. nginx gave up waiting — for the response, or for the connection itself if proxy_connect_timeout ran out first.

        proxy_connect_timeout 5s;    # time to establish the connection
        proxy_send_timeout   60s;    # max gap between two writes to the upstream
        proxy_read_timeout   60s;    # max gap between two reads from the upstream

The temptation is to raise proxy_read_timeout until the 504s stop. That converts a visible failure into a slow one, and holds a worker and a client connection open for the duration. Raise it when the work genuinely takes that long — a report, an upload — and otherwise find what is slow.

Worth knowing: the upstream usually carries on working after nginx has given up. The client is told it failed while the database query is still running, which is why a retry can make things worse rather than better. If there are retries above this proxy, work out what they actually total — the schedule has to fit inside proxy_read_timeout, and it frequently does not.

502 vs 504 is the distinction in one page, and 504 Gateway Timeout has the causes.

499 — the client left first

Not a standard code. nginx logs it and never sends it; nobody receives a 499.

A rise in 499s almost always means you got slower, not that clients changed. Read it against upstream response time rather than treating it as a client-side problem.

If you want to know which of the three you are looking at and who most likely produced it, the HTTP status explainer answers that specifically.

TLS

Certbot writes a working config for you, and the part worth understanding is the redirect:

sudo certbot --nginx -d example.com -d www.example.com

server {
    listen 80;
    server_name example.com www.example.com;
    return 301 https://$host$request_uri;
}

Two cautions. Test renewal before you need it — sudo certbot renew --dry-run — because a renewal that fails silently becomes an outage in ninety days with no warning. And if you terminate TLS here, make sure X-Forwarded-Proto is set, or your application will generate http:// URLs for an HTTPS site.

Load balancing across several upstreams

upstream app {
    server 127.0.0.1:3000 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:3001 max_fails=3 fail_timeout=30s;
    keepalive 32;
}

server {
    location / {
        proxy_pass http://app;
        proxy_http_version 1.1;
        proxy_set_header Connection "";   # required for keepalive before nginx 1.29.7
    }
}

max_fails and fail_timeout tune how quickly nginx stops sending traffic to a backend that is failing. They apply even when you leave them out — one failure takes a server out for ten seconds — so max_fails=3 here makes nginx more patient with a flaky backend, not less. Set them when the defaults are wrong for you, not to switch the behaviour on.

On nginx before 1.29.7 the Connection "" line is easy to miss and costs real latency when omitted — keepalive to upstreams does not work without it. From 1.29.7, including the 1.30 stable branch, nginx uses HTTP/1.1 and upstream keepalive by default and stops sending the Connection header, so the line is harmless but no longer required.

Rate limiting, carefully

limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;

server {
    location /api/ {
        limit_req zone=api burst=20 nodelay;
        proxy_pass http://app;
    }
}

burst is what makes this usable. Without it, a browser loading ten assets at once is rate limited on a legitimate page view. With nodelay, bursts are served immediately rather than queued.

Set it in a test environment and watch it before enabling in production. A rate limit that is too tight looks exactly like an outage, and it will be blamed on the application.

Caching, and when not to

proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=static:10m
                 max_size=1g inactive=60m;

location /static/ {
    proxy_pass http://app;
    proxy_cache static;
    proxy_cache_valid 200 60m;
    add_header X-Cache-Status $upstream_cache_status;
}

The X-Cache-Status header is worth having from the start, because "is this cached" is otherwise guesswork.

Do not cache authenticated responses without being extremely deliberate about the cache key. Serving one user's page to another is the worst bug in this entire article, and it is easy to cause.

Before you reload

sudo nginx -t          # validate — always, every time
sudo systemctl reload nginx

reload finishes in-flight requests with the old workers rather than dropping them, which restart does not. And nginx -t costs nothing: a reload with a broken config is refused and nginx quietly carries on with the old one, so without the test you may not notice your change never applied — and the next restart is what takes the site down.

One more thing worth knowing: add_header at one level replaces every inherited add_header rather than adding to it. A location that sets its own Cache-Control silently drops the security headers defined at server level. If you use both, repeat them — or include them from a snippet in every location that declares one. On nginx 1.29.3 and later, including the 1.30 stable branch, add_header_inherit merge; makes a level add to the inherited headers instead of replacing them.