A load balancer or reverse proxy sits between the client and the application. When the proxy cannot complete the request, it returns a 5xx on behalf of the upstream. The three codes look similar from the browser and mean very different things to the operator.
502 Bad Gateway
The proxy received an invalid or incomplete response from the upstream. It connected, but what came back was not usable.
Common causes:
- The application crashed mid-response.
- The upstream closed the connection without sending headers.
- A protocol mismatch: the proxy speaks HTTP/1.1, the upstream expects HTTPS or HTTP/2.
- An upstream that returned malformed headers.
- A container that restarted during the request.
Where to look:
- Proxy error log (
error.login nginx,proxy_error_login Envoy). - Upstream application log at the same timestamp.
- Whether the upstream process is running and listening on the configured port.
- TLS settings:
proxy_pass https://versus an HTTP listener. - Response size and timeout settings.
A 502 often means the application died. The proxy is reporting the symptom, not causing it.
503 Service Unavailable
The server is not able to handle the request right now. It may be overloaded, starting up, draining, or in maintenance. The response may include Retry-After.
Common causes:
- All workers busy, queue full.
- Deployment in progress, health checks failing.
- Circuit breaker open.
- Rate limiting at the edge.
- Deliberate maintenance mode.
Where to look:
- Saturation metrics: CPU, memory, connection count, queue depth.
- Health-check status and recent deploys.
- Autoscaling events.
- Whether the load balancer removed the instance from rotation.
A 503 is often correct behavior: the server is protecting itself. The fix may be capacity, not code.
504 Gateway Timeout
The proxy waited for the upstream and gave up. It connected, sent the request, and did not receive a complete response within the configured timeout.
Common causes:
- A slow database query.
- An external API call without a timeout.
- A large report generated synchronously.
- A deadlock or lock contention.
- Network latency between proxy and upstream.
- A timeout set too low for a legitimately slow endpoint.
Where to look:
- Upstream request duration metrics, especially p95 and p99.
- Slow query logs.
- Whether the upstream has its own timeout that is longer than the proxy's.
- Whether the endpoint is synchronous work that should be a job.
The 504 trap
A 504 means the proxy stopped waiting. It does not mean the upstream stopped working. The request may still be running and may still commit its side effect. Retrying a POST after a 504 can duplicate the operation.
Mitigations:
- Idempotency keys on writes.
- Make long-running work asynchronous: return
202 Acceptedwith a job ID. - Set timeouts deliberately at every layer, with the outer layer longer than the inner one.
- Log a request ID at the proxy and the application so the two can be correlated.
Timeout layering
A common misconfiguration is a proxy timeout shorter than the application's own timeout. The proxy gives up, the application continues, and the client sees a 504 while the work completes. The reverse is also bad: the application times out first and returns a 500 that the proxy passes through.
Set a budget: client timeout > proxy timeout > application timeout > upstream dependency timeout. Every layer should fail before the layer above it, so the error is reported where it happened.
A triage sequence
- Identify which component returned the error from its headers (
Server,Via, vendor-specific IDs). - Correlate the timestamp across proxy and application logs.
- Determine whether the upstream received the request. If it did not, the problem is connection-level (502/503). If it did, the problem is duration or response validity (504/502).
- Check recent deploys, config changes and traffic spikes.
- Reproduce with a single request while watching upstream logs.
What not to do
- Do not retry all 5xx blindly; respect idempotency.
- Do not raise timeouts without finding the slow dependency; you will just hold connections longer.
- Do not return 200 with an error body to “fix” the dashboard. The error is real.
- Do not assume the upstream is down because the code is 502. It may be up and returning garbage.
The 502 page compares all three codes with example responses. If you are choosing a code for your own service, the status code reference covers the decision.