foley.sources.resilience#

Graceful degradation for HTTP source adapters — throttle · backoff · circuit-break (#12).

resilient() wraps the injectable Transport DI seam with three composed policies so a flaky or rate-limited remote source degrades predictably instead of hammering the API or crashing a run:

  • throttle — a token bucket honouring the source config’s declared rate (per_min / per_day) so foley never exceeds the published limits;

  • backoff — exponential retry on 429 / 5xx (and transport exceptions), honouring a Retry-After header when present;

  • circuit-break — after N consecutive failures the breaker opens and fast-fails with SourceUnavailable for a cool-down window (no thundering herd).

All timing goes through injected clock / sleep callables (default time.monotonic / time.sleep) so the whole thing is deterministically testable with a fake clock and a no-op sleep — no real network, no real waiting. Stdlib-only.

Module Attributes

SECONDS_PER_DAY

Seconds in a rolling rate-limit day (the per_day window length).

Functions

make_resilient_transport_from_config(config, *)

Build a resilient transport from a source config (its rate drives the throttle).

resilient(transport, *[, rate, retry, ...])

Wrap a Transport with throttle + backoff + circuit-break.

Classes

BreakerPolicy([fail_threshold, reset_timeout_s])

Circuit-breaker thresholds.

RetryPolicy([max_attempts, retry_on, ...])

Exponential-backoff retry policy for transient HTTP failures.

Exceptions

SourceUnavailable

Raised when a source's circuit breaker is open or its retries are exhausted.
class foley.sources.resilience.BreakerPolicy(fail_threshold=5, reset_timeout_s=30.0)[source]#

Bases: object

Circuit-breaker thresholds.

class foley.sources.resilience.RetryPolicy(max_attempts=4, retry_on=(429, 500, 502, 503, 504), base_delay_s=0.5, max_delay_s=30.0, respect_retry_after=True)[source]#

Bases: object

Exponential-backoff retry policy for transient HTTP failures.

foley.sources.resilience.SECONDS_PER_DAY: float = 86400.0#

Seconds in a rolling rate-limit day (the per_day window length).

exception foley.sources.resilience.SourceUnavailable[source]#

Bases: RuntimeError

Raised when a source’s circuit breaker is open or its retries are exhausted.

foley.sources.resilience.make_resilient_transport_from_config(config, *, base=None, **inject)[source]#

Build a resilient transport from a source config (its rate drives the throttle).

Parameters:
Returns:

A resilient transport wrapping base.

foley.sources.resilience.resilient(transport, *, rate=None, retry=None, breaker=None, clock=<built-in function monotonic>, sleep=<built-in function sleep>, resend_after_transport_error=True)[source]#

Wrap a Transport with throttle + backoff + circuit-break.

Parameters:

  • transport – The base transport callable (method, url, *, params, headers, json).

  • rate (Optional[dict]) – The source config’s rate dict ({'per_min':…, 'per_day':…}).

  • retry (Optional[RetryPolicy]) – The RetryPolicy (default: 4 attempts on 429/5xx).

  • breaker (Optional[BreakerPolicy]) – The BreakerPolicy (default: open after 5 consecutive fails).

  • clock (Callable[[], float]) – Monotonic time source (injected for tests).

  • sleep (Callable[[float], None]) – Blocking sleep (injected for tests).

  • resend_after_transport_error (bool) – Retry a non-GET request after a transport-level failure (timeout, reset). False for a paid source: the provider may have accepted — and billed — the request before the connection failed, so re-sending would be an unauthorized second paid call. Status-coded rejections (429 / 5xx) are still retried: those are answers, not charges.

Returns:

A transport with the same signature, plus a .reset() method. Raises SourceUnavailable when the breaker is open or retries are exhausted.