Stop waiting for slow dependencies to time out or you will kill your own service.

A complete outage is actually easier to handle than a slow response. When a service is down, the connection is refused in milliseconds, and your worker threads are freed immediately. But when a service lags—say, jumping from 40ms to 30s—your system keeps those threads, database connections, and memory slots open for the full duration. Because these resources are finite, a single slow dependency can saturate your entire thread pool, taking down unrelated pages or services that don't even call that dependency.

Why slow calls are more dangerous than crashes

Every outbound request holds a set of resources hostage. If you have a fixed pool of 200 worker threads and 100 of them are hanging on a slow payment API, you've just lost 50% of your capacity. If the request volume stays steady, you hit 100% saturation in seconds. This is why "failing fast" is the only way to maintain system stability.

Set two types of timeouts for every call

You cannot rely on default library timeouts; they are usually way too long. You need to explicitly configure two different values for every remote call:

1. Connection Timeout: How long you wait to establish the TCP socket. This should be very short (e.g., 500ms to 2s) because if the handshake doesn't happen quickly, the network or the server is likely dead.
2. Request Timeout: How long you wait for the actual data payload after the connection is established.

To pick these numbers, don't guess. Look at your P99 or P99.9 latency metrics for the dependency. If 99.9% of calls finish in 200ms, setting a timeout at 300ms ensures you only cut off 0.1% of potentially successful calls while protecting yourself from the 30-second hangs that crash your site.

Implementing a Circuit Breaker

A timeout handles a single slow call, but a circuit breaker prevents you from making the call in the first place once a failure threshold is hit. It acts as a proxy that monitors for errors.

• Closed State: Everything is normal. Requests flow through.
• Open State: The error rate exceeds a threshold (e.g., 50% failure over 10 seconds). The breaker "trips," and all subsequent calls fail immediately without even trying to hit the network. This gives the dependency room to recover.
• Half-Open State: After a sleep window (e.g., 30 seconds), the breaker lets a few "probe" requests through. If they succeed, it closes the circuit; if they fail, it opens again.

If you are using Java/Spring, Resilience4j is the standard here. In Go, you might use a library like gobreaker.

Handling the "Fallback"

The real value of a circuit breaker is what you do when the circuit is open. Instead of returning a 500 error to the user, implement a fallback strategy:

• Cache: Return the last known good value from Redis.
• Default: Return a static "Service temporarily unavailable" message or a default product price.
• Degrade: If a "Recommended Products" service is slow, just hide that section of the UI entirely so the rest of the page loads.

By combining aggressive timeouts with a circuit breaker, you transform a fatal system-wide hang into a graceful degradation of a single feature.