Process · 16 of 19 · Preflight

Preflight: score every agent change against incidents you have already solved

Hangar's own dead-ends list is the corpus: each case has a documented cause and a known fix, so the verifier is a deterministic check and not another model. Any change to a profile, a model or a policy has to score no worse than the last one.

Preflight: score every agent change against incidents you have already solved Process diagram of an evaluation harness in which a seed corpus of documented incidents is applied as a broken fixture in a disposable namespace inside a Glidepath PipelineRun, the agent under test works through Clearance, a deterministic verifier scores the result, and a regression gate blocks profile, model or policy changes that score worse than the baseline; a table lists six seed cases with their known root causes and verifiers. GLIDEPATH PIPELINERUN · ONE PER CASECASETASKDIFFSCOREBASELINESeed corpusdocumented dead endsBroken fixtureAgentRun · TTLAgent under testprofile: preflightVerifierdeterministic checksScorecardPrometheus · TowerRegression gateblocks profile · model · policySEED CORPUS · GROUND TRUTH FROM DOCUMENTED DEAD ENDSCASEKNOWN ROOT CAUSEDETERMINISTIC VERIFIERCanary broken by image tagbad image tag in gitops valuesAnalysisRun passes; diff limited to values.yamlLiveness probe never Readyprobe path pointed at /readinesspod Ready; probe is /actuator/health/livenessRabbitMQ broker never ReadyRabbitMQ 4.1 with operator 2.23RabbitmqCluster Ready; broker is 4.2.xComposite never goes Readyauto-ready: no Ready condition on CRXR Ready; composition sets ready annotationXRD change silently revertedhand-apply reverted by ArgoCD selfHealfix lands as a commit; no kubectl apply loggedImage scan fails the buildtransitive CVE in Java dependenciesimage-scan gate green; pom diff is pins onlyReal incidents with a known cause and a known-good fix, so no judge model is needed. Other agent types get their own suites.LEGENDJudge of recordUnder testDisposableHistoryScore

Principle honored

Live verification: prove against real state, and never trust a pass that could mean the gate is off. Each case also has a seeded negative run that must fail.

What changes

  • Runs as a Glidepath pipeline, so it inherits provenance, signing and results archival.
  • Scored on outcome, blast radius (paths touched vs allowlist), Clearance denials, retries and cost.
  • Gate lives on the Clearance repo itself: profile, model and policy changes are PRs that must pass Preflight.

Start small

Proposal, not built. Start with two cases (the probe path and the auto-ready bug), because they are one-file fixes with unambiguous verifiers, and add the rest once the harness is trusted.

Hangar · Autopilot