Retry Logic with Injected Sleep Backoff

def retry_call(dependency, sleep):
    max_attempts = 7
    failure_threshold = 3
    reset_timeout_seconds = 15
    attempts = 0
    state = "closed"
    fail_count = 0
    while True:
        if state in ("closed", "half_open"):
            if attempts >= max_attempts:
                raise RuntimeError("circuit breaker: max attempts exhausted")
            attempts += 1
            try:
                return dependency()
            except Exception:
                if state == "closed":
                    fail_count += 1
                    if fail_count >= failure_threshold:
                        state = "open"
                else:
                    state = "open"
                continue
        else:
            if attempts >= max_attempts:
                raise RuntimeError("circuit breaker: max attempts exhausted")
            sleep(reset_timeout_seconds)
            state = "half_open"

{
  "outcome": "success",
  "final_attempts": 5,
  "delay_sequence": [
    15,
    15
  ]
}

{
  "failure_threshold": 3,
  "reset_timeout_seconds": 15,
  "max_attempts": 7
}

A web application checks out a connection from a pooled connection manager in front of a relational database. During a failover, the pool starts returning connection errors on nearly every checkout; a circuit breaker trips after a few consecutive failures so the application stops hammering the dying pool, waits, and periodically sends a single probing checkout to see whether the failover has completed.

[
  "fail",
  "fail",
  "fail",
  "fail",
  "succeed"
]

Circuit breaker: open the circuit after 3 consecutive failures, wait 15 seconds before each half-open trial, up to 7 real attempts total.