Hangar

Airframe · Scorecard

Airframe, measured

Autopilot only works if an agent can change an app safely without guessing. So I score Airframe, the contract every app is built on, against what an agent needs: ten dimensions, fourteen acceptance checks, all automated. A+ means 97 or better and all fourteen checks. The number starts low on purpose, because it measures the finished contract, and it moves only when the work is real.

Score now97.3/100Oct 29, 2026 · grade A+
Acceptance checks14/14from 1 at the baseline
Since the baseline+70Sep 26, 2026 to Oct 29, 2026
A+ bar97 and 14/14certified

Progress over time

Each filled point is a committed run of the tool. The hollow points are the roadmap's milestone targets still ahead, not measurements.

Airframe scorecard over time Baseline September 26, 2026: 27.3 out of 100, 1 of 14 checks; M0 September 26, 2026: 41.3 out of 100, 2 of 14 checks; M1 and M2 September 29, 2026: 72.4 out of 100, 7 of 14 checks; Review fixes October 9, 2026: 84 out of 100, 10 of 14 checks; AppSpec in the contract October 9, 2026: 86.6 out of 100, 11 of 14 checks; Cloud targets and the rebuild October 25, 2026: 89.5 out of 100, 11 of 14 checks; airframe.* tools behind Clearance October 28, 2026: 97.1 out of 100, 13 of 14 checks; A+: fromService October 29, 2026: 97.3 out of 100, 14 of 14 checks. Targets ahead: . 0 25 50 75 100 A+ at 97 27.3 Baseline Sep 26, 2026 · 1/14 41.3 M0 Sep 26, 2026 · 2/14 72.4 M1 and M2 Sep 29, 2026 · 7/14 84 Review fixes Oct 9, 2026 · 10/14 86.6 AppSpec in the contract Oct 9, 2026 · 11/14 89.5 Cloud targets and the rebuild Oct 25, 2026 · 11/14 97.1 airframe.* tools behind Clearance Oct 28, 2026 · 13/14 97.3 A+: fromService Oct 29, 2026 · 14/14
  1. Baseline · Sep 26, 2026 Before any of the A+ work. One acceptance check passed: every live values file rendered.
  2. M0 · Sep 26, 2026 The chart guard, chart tests and CI, AGENTS.md and a first airframe validate. M0 closed the next day against a target of 35.
  3. M1 and M2 · Sep 29, 2026 A strict schema, component outputs, a contract bundle, the release-file split, an ownership gate, field-level agent scope and replayable walkthroughs.
  4. Review fixes · Oct 9, 2026 Fixes from the architecture review: AGENTS.md on every component, an owner and a risk class on every values field, status conditions and reason codes on 17 of 20 XRDs, and every XRD described and in the catalog. Past the M2 target of 75 for the first time.
  5. AppSpec in the contract · Oct 9, 2026 The AppSpec schema (one document stating an app and its environments) moved into Airframe's contract bundle, where an agent reading the contract finds it. Interaction surface doubled; the airframe.* tools are what it still lacks.
  6. Cloud targets and the rebuild · Oct 25, 2026 Three new XRDs (23 in all), cross-field CEL rules on 22 of them, fromComponent in nearly every env entry (2 literals left of 78), 23 chart guards and 48 chart tests, seven walkthroughs. Past the M3 target of 88, though M3 itself (the planner and airframe.* tools) is not built. The chart README got shorter when it was rewritten to describe today's chart, which cost a little discoverability.
  7. airframe.* tools behind Clearance · Oct 28, 2026 Seven read-only airframe.* tools (capabilities, validate, explain, plan, status, describe, verify) behind Clearance over a GET-only Kubernetes reader, live on hangar-dev through MCP (autopilot#19). The planner's cicd.yaml patch now passes Glidepath's agent scope. The schema marks its 12 deliberate passthroughs, so every other object is strict (airframe#128). The scorecard itself got more honest (hangar#94): tools are counted from what Clearance registers rather than a file name, the extraManifests deny is measured instead of hard-coded, and typed maps and if/then fragments are no longer miscounted as open objects. One check left: two cross-app URLs that wait on fromService.
  8. A+: fromService · Oct 29, 2026 The last derived-name literals are gone: env[].fromService (airframe v0.3.168) builds another app's Service URL from the namespace convention the platform now passes every environment, so an agent never guesses it. All 14 acceptance checks pass and the overall clears 97: the A+ bar. The milestone it measures readiness for, the parachute sentence running live through Clearance, is still ahead (airframe.apply, the agents' GitHub App, AgentRun).

Ten dimensions, measured October 29, 2026

The bar is the latest score; the thin mark on it is where the baseline stood, and the amber tick is the A+ line.

DimensionScoreHistory What the measurement foundWhat A+ still needs
1 Discoverability 97 35 → 50 → 84 → 100 → 100 → 97 → 97 → 97 AGENTS.md on all 23 components and every XRD in the catalog; the chart README is down to 170 lines a chart README that documents every values key again
2 Schema precision 91 45 → 45 → 76 → 81 → 81 → 86 → 91 → 91 every schema node is described; all 67 non-passthrough objects strict, 12 passthroughs marked; 22 of 23 XRDs carry a cross-field CEL rule tighter leaf fields (78 of 206 constrained), examples
3 Component contracts 100 4 → 3 → 68 → 77 → 76 → 99 → 99 → 100 every component declares outputs and verify checks; no env entry hard-codes a derived name (fromComponent, fromService) held; AF-SVC-001 flags a hand-written in-cluster URL
4 Pre-merge validation 100 22 → 76 → 94 → 96 → 96 → 99 → 99 → 100 airframe validate is a required check, with 23 chart guards that say what to fix the last two chart guards
5 Write safety 100 13 → 25 → 67 → 100 → 100 → 100 → 100 → 100 every values field carries an owner and a risk class; no live file mixes owners held; keep machine-owned data in the release file
6 Observe and verify 94 21 → 21 → 57 → 79 → 79 → 80 → 94 → 94 20 of 23 XRDs declare status conditions and a closed set of reason codes the last three XRDs; describe and status tools are live
7 Docs for agents 100 39 → 54 → 100 → 100 → 100 → 100 → 100 → 100 seven walkthroughs replay live, and every YAML block in the user docs parses held; every new component ships its walkthrough
8 Interaction surface 91 27 → 27 → 27 → 27 → 55 → 55 → 91 → 91 seven read-only airframe.* tools behind Clearance, live through MCP; the planner's output passes Glidepath's agent scope airframe.apply as the agents' own GitHub App (AF-8b), and a config API beside Tower's UI
9 Safety integration 100 20 → 20 → 50 → 80 → 80 → 80 → 100 → 100 every values field is risk-classed and field-level agent scope is built into Clearance held; extraManifests and the other escape hatches stay denied to agents
10 Hygiene and determinism 100 46 → 92 → 100 → 100 → 100 → 100 → 100 → 100 helm lint, 48 chart tests and chart CI; all 17 live values files render held; keep the fleet rendering clean
Overall 97.3 27.3 → 41.3 → 72.4 → 84 → 86.6 → 89.5 → 97.1 → 97.3 14 of 14 acceptance checks · A+ needs 97 and 14 of 14

The fourteen acceptance checks

A+ needs every one. A high average cannot hide a missing check.

CheckBaselineSep 26, 2026M0Sep 26, 2026M1 and M2Sep 29, 2026Review fixesOct 9, 2026AppSpec in the contractOct 9, 2026Cloud targets and the rebuildOct 25, 2026airframe.* tools behind ClearanceOct 28, 2026A+: fromServiceOct 29, 2026
typo acceptance is 0% ··✓✓✓✓✓✓
schema description coverage >= 95% ···✓✓✓✓✓
every non-passthrough object rejects unknown keys ······✓✓
contract bundle published ··✓✓✓✓✓✓
AGENTS.md at root and on every component ···✓✓✓✓✓
all component XRDs declare outputs and verify checks ···✓✓✓✓✓
no derived-name literals in live env entries ·······✓
airframe validate exists and is a required check ·✓✓✓✓✓✓✓
no live file mixes human and machine-owned keys ··✓✓✓✓✓✓
shared base layer in the ApplicationSets ··✓✓✓✓✓✓
airframe.* tools exist ······✓✓
AppSpec schema exists ····✓✓✓✓
executable walkthroughs replace prose quickstarts ··✓✓✓✓✓✓
every live values file renders cleanly ✓✓✓✓✓✓✓✓

Principle honored

Measured, not felt. Every check is automated, so A+ is something the tool prints, not something I claim. Every run that moves the number is committed, so the history on this page is the tool's own output.

Corrections to the tool

The first run on September 29, 2026 said 58. The tool was looking for component contracts and for Clearance where they were first planned to live, not where M1 and M2 put them, and two checks were hard-coded to zero. I fixed the paths and turned those two checks into real ones.

The first run on October 9, 2026 said 82.2, because the app repos had renamed platform/ to glidepath/ and the release-file layering had moved into Glidepath, so the tool lost both. The release-file split looked undone when it wasn't. I pointed the tool at the new paths. Each fix is in the same commit as its score, and snapshots before it were not re-scored.

The run on October 25, 2026 needed two more. The dev cluster's repo had been renamed in the October rebuild, so I pointed the tool at both names before running it. Then it said 89.2, because it counted one docs block as broken YAML when the block simply held two documents. The tool now accepts that, which made it 89.5.

Limits

The scorecard can't tell whether an agent would actually succeed. The parachute sentence does: one request to an agent to set up a new Python app named parachute, with dev and test, staging and prod, a canary step and an env var, run as a Preflight case that passes only when the result is right. I use both.

Re-run it yourself: python3 tools/airframe-scorecard/scorecard.py --json out.json. The tool, every committed run, the rubric and the roadmap are on GitHub. More on Autopilot on the AI workloads page.