SLOs · 04 of 04 · One poll tick

What happens in one poll tick

One scheduled task, one cluster at a time. The expensive parts, listing and querying, are batched per cluster, and the part that must never repeat, announcing, is guarded by an atomic claim.

What happens in one poll tick Top row, left to right: a scheduler task runs every 120 seconds under a global lock, lists the SLOs on each cluster through the Kubernetes API, queries Prometheus with three batched queries per cluster through the apiserver service proxy, and classifies each of three signals per SLO with hysteresis. When a state changed, it flows down to a Claim step, a compare-and-set in Postgres, then to Announce, which sends a Backstage notification, and then to Tower, which shows it in Recent Activity under the SLOs filter. A dashed return path from Announce back to Claim restores the claim when the send fails. POLL · EVERY 120 SECONDSTRANSITION · CLAIM, THEN ANNOUNCESTATE CHANGEDSEND FAILED · RESTOREScheduler taskevery 120s · global lockList SLOseach cluster · K8s APIQuery Prometheus3 batched queries / clusterClassifyhysteresis · 3 signals per SLOClaimcompare-and-set in PostgresAnnouncenotifications.sendTowerRecent Activity · SLOsNothing is sent unless a stored state actually changed.LEGENDStepThe dedupe pointChange pathRestore on failure

Batched

Three queries per cluster: period burn, 5-minute burn and 1-hour burn, matched back to each SLO by label. Cost no longer grows with the number of SLOs.

One runner

The scheduler runs the task on one replica at a time. The claim covers the remaining case, a timed-out run still going when the next one starts.

Who gets it

Everyone, by default. Set recentActivity.slo.recipients to owner to send to the app's catalog owner instead; an app with no owner still broadcasts.

Hangar · SLOs