Docs · Hangar

Airframe A+ program: a service catalog an AI agent can operate

Goal: an agent (Claude Code, a workload, a client) can discover, configure, validate, change and verify anything Airframe manages, using only machine-readable contracts and governed tools, and can tell when it has failed and why. Success is measured, not felt: tools/airframe-scorecard/scorecard.py.

The test we are building toward is one sentence typed into a Claude Code session:

provision a new python application named parachute. give it a dev and test ground environment. a staging and prod flight environment. and configure it with 1 100% weight canary step, an URL=http://myendpoint.io env var.

This is the parachute sentence, named after the app it asks for. The rest of the Autopilot docs and diagrams use that name for it, and “the parachute test” means running it end to end.

Status labels used here: Built, Draft (written, not applied), Proposal, Unverified.

Progress (2026-09-26, end of M0): 41.3/100, 2 of 14 acceptance checks. Shipped in Airframe v0.3.91: the rollout guard (AF-10a), render tests and chart CI (AF-10b), AGENTS.md (AF-1a), tools/airframe-validate v0 (AF-4a). Validation moved 22 -> 76 and hygiene 46 -> 92. The scorecard’s new baseline is tools/airframe-scorecard/baseline-2026-09-26b.json. One dimension went down: component contracts 4.0 -> 3.4, because baggage-api’s env file added four hand-written derived names (bag-mq-connection, bag-mq-user-credentials), which is the gap AF-3 (fromComponent) closes. Note: the values schema still accepts unknown keys; strictness lives in airframe-validate until AF-2 generates a strict schema. Unknown fields in an XR are silently pruned by ArgoCD (U1), so XR files need the same check.

1. Baseline (2026-09-26)

Measured by the scorecard tool against the real repos. It scores readiness against the full A+ contract, so it is a progress meter: most of the gap is artifacts that do not exist yet, not things that are broken. It is stricter than a “does it work with a careful agent” judgment, which is why it reads lower than an eyeball grade would.

Re-run any time: python3 tools/airframe-scorecard/scorecard.py --json out.json. The committed baseline is tools/airframe-scorecard/baseline-2026-09-26.json.

Two findings from running the chart, not reading it

1. Typos pass. rolout:, replcas: 5 and totallyMadeUp render with exit 0 and are ignored. A component with size: gigantic (the Redis XRD allows small, medium, large) also renders; it fails only after merge, on the cluster, or is pruned silently.
2. Configuring before the first image renders a broken Rollout. Setting rollout.steps and env with no image renders image: ':'. The quickstart works around it with rollout: null, but then you cannot configure the rollout. An agent declaring everything up front, which is the natural thing to do, hits this on the first environment. autopilot/tests/test_airframe_plan.py::test_CANARY_the_chart_bug_the_workaround_exists_for pins the behaviour and will fail on purpose when the chart is fixed.

2. What A+ means

A+ = overall ≥ 97 and all 14 acceptance checks pass. Each check is automated by the scorecard.

Beyond the scorecard, the end-to-end acceptance test is the parachute sentence, run as a Preflight case with a seeded bad run, and a resume test (kill the session mid-plan, resume by task_id).

3. The design: the values contract is the API

A NodeJS or Python application’s ongoing configuration is its values files, not its XR. So “making Airframe AI-friendly” means making that contract a first-class, machine-readable, validated, ownership-aware API, and giving agents a small verb set over it: describe, explain, plan, validate, apply (as a PR), status, verify.

Glidepath’s cicd.schema.json is the model: additionalProperties: false, long descriptions, explicit oneOf shapes. The Airframe schema should read like it.

One source, generated artifacts

XRD OpenAPI + chart values schema + sidecar meta + dead-end rules
                            │
                    contract generator (CI)
                            │
                  airframe-contract.json   ← versioned with each Airframe tag
        ┌───────────┬──────────┼──────────┬──────────────┐
   airframe     Tower forms   airframe.*   docs, AGENTS.md,
   validate     + validator   tools (MCP)  reference (generated)

Diagram: diagrams/plan/05-contract-architecture.html.

Sidecar meta. A Kubernetes CRD’s structural schema rejects unknown x- keys, so the extra vocabulary cannot live in an XRD. It lives in contract/<kind>.meta.yaml next to it, keyed by JSON pointer, and the generator merges it. In the chart’s values.schema.json (not a CRD) the same keys can be inline annotations; Unverified: that Helm’s validator ignores unknown keywords. Check with helm lint before relying on it.

A sample is in drafts/airframe/contract/redis.meta.yaml.

4. The ten workstreams

Effort is for one person. Born-A+ rule: every component built from now on (MongoDB, OAuth, nginx, the chart’s agent block) ships with outputs, verify checks, sidecar meta, AGENTS.md and a description on every field. Nothing new adds to the retrofit.

AF-1 Discoverability (35 → A+)

AF-1b done: jfillman/airframe#10 (2026-09-27). llms.txt, contract/airframe-contract.json, hangar.io/agent-summary on all 13 XRDs, and airframe.capabilities. 10/10 real capability questions answered from the bundle alone and cross-checked against source (tools/test_capabilities_coldstart.py). Still open: add-to-catalog review for all 13 (a real Backstage-template decision, left for the user), and wiring the 10-question proof as a registered case in the separate autopilot repo’s own Preflight harness. AF-1a (AGENTS.md) shipped in M0.

• airframe/AGENTS.md (root) and compositions/<x>/AGENTS.md: where the contract lives, how to validate, what not to touch. Draft: drafts/airframe/AGENTS.md.
• airframe/llms.txt: an index of the contract and docs.
• hangar.io/agent-summary annotation on every XRD; add-to-catalog reviewed for all 13.
• contract/airframe-contract.json and the airframe.capabilities tool.
• Accept: a cold-start Preflight case answers 10 capability questions from the contract alone.
• Effort: 2 days after AF-2’s generator exists.

AF-2 Schema precision (45 → A+)

Chart-level values schema: done, jfillman/airframe#6 (2026-09-27). Descriptions 5% -> 100%, strictness 100% (baked into values.schema.json directly, not a runtime pass), values.yaml generated, components[] discriminated against real XRDs, mutation test 48/48 on the live fleet. Still open: the XRD-level half (descriptions/CEL rules, still at 78/125 described) and the CI-required-check widening to app/tenants repos (that part is AF-4b).

• Schema-first: move the 548 lines of prose from values.yaml comments into values.schema.json descriptions, then generate values.yaml (defaults and doc comments) from the schema so they cannot drift.
• additionalProperties: false on every object except marked passthroughs (podSpec, extraManifests, canaryAnalysis, blueGreen).
• Enums, patterns (DNS names, quantities), examples, defaults. Coverage gates in CI: descriptions ≥ 95%, strictness 100%.
• XRDs: CEL rules for cross-field constraints (x-kubernetes-validations), and required descriptions.
• Discriminated union for components[] on type, each arm generated from that component’s XRD spec, so size: gigantic fails before merge. (Tower’s hand-rolled validator has no oneOf; validate server-side or adopt a real validator.)
• Accept: a mutation test over every live values file (add a key, misspell a key, break an enum) is rejected 100%.
• Rollout: warn-only first, enforce after the fleet baseline sweep passes.
• Effort: 1 week. Risk: strictness can break an existing environment, hence the sweep.

AF-3 Component contracts (4 → A+)

Redis/PostgreSQL/RabbitMQ: done, jfillman/airframe#8 (2026-09-27). Sidecar meta + fromComponent + AF-COMP-002 lint, verified against real Compositions; a real migration PR staged against boarding-api (unsigned - awaiting the user’s own commit). Still open: SecretStore, and status.outputs published live on the XR itself (needs XRD schema + Composition patches, cluster-verification-heavy).

• Each component declares outputs in its sidecar meta: Secret and ConfigMap names, keys, service DNS, ports.
• Chart feature fromComponent: env: [{name: REDIS_URL, fromComponent: {name: cache, output: url}}], resolved to a literal or a valueFrom reference. Agents stop guessing cache-master.
• Components publish status.outputs so it is discoverable at runtime.
• Lint AF-COMP-002: every reference resolves to a declared component output.
• Accept: zero derived-name literals across the fleet; every reference resolves.
• Effort: 1 week for Redis, Postgres, RabbitMQ, SecretStore; MongoDB and OAuth are born with it.

AF-4 Pre-merge validation (22 → A+)

Partial: jfillman/airframe#9 (2026-09-27). L1/L2 plus five dead-end rules (AF-CLUSTER-001, AF-ENV-001, AF-COMP-001, AF-SECRET-001, and an advisory AF-COMP-003 that fires on the fleet’s remaining hand-written component references - real, not hypothetical). --format json added. Still open: L3 kubeconform, SARIF, the remaining seed rules below that need cluster/build context, and required-check widening from gitops-only to app + tenants repos. airframe validate is one tool with six layers. It runs as a required check on app repos (platform/envs), gitops repos, and the tenants repo (xr-requests), and as an agent tool.

Output is JSON and SARIF with a rule id, a JSON path and a fix hint, so an agent can retry against it.

Seed rules, each with a seeded failing fixture (the dead-ends list becomes lint):

• Accept: every dead end has a rule and a failing fixture; every live file passes; under 10 seconds.
• Effort: 1 week for L1 to L3 as a container and a Tekton task; the rules accrue.

AF-5 Write safety (13 → A+)

• One file, one owner. Ground: platform/envs/<env>.yaml (human) and <env>.release.yaml (the deploy stage). Flight: values.yaml (human) and values.release.yaml (Glidepath’s release PR). ApplicationSets layer base → env → release, later wins.
• A shared base layer: platform/base.yaml and gitops-<app>/base/values.yaml, so “set this everywhere” is one edit.
• Field ownership and risk in the sidecar meta; an ownership gate (AF-OWN-001).
• agent-scope becomes field-aware (JSON-pointer allow and deny), not path-only. extraManifests, rollout.image, releaseTracking, networkPolicy and httpRoute are denied to agents by default.
• Canonical formatting and a comment-preserving patch engine, so agents do not drop the comments that carry meaning.
• Accept: no live file mixes owners; a property test shows a release PR and a config PR never conflict.
• Coordination: three repos change together (airframe, glidepath’s open-release-pr and deploy stage, Tower’s Config tab). Unverified: that ArgoCD’s git files generator can exclude *.release.yaml so it does not create a second Application; the generator supports exclude, but test it on a scratch app first.
• Effort: 2 weeks with migration.

AF-6 Observe and verify (21 → A+)

AF-6a done: jfillman/airframe#11 (2026-09-27). Redis/PostgreSQL/RabbitMQ each set a custom ComponentReady condition (a closed reason set, not Crossplane’s own Ready) computed from the composed child’s real status - live-verified via copy-composition on every branch. Still open: verify contracts (x-hangar-verify) beyond the verify: list already in each xrds/<type>.meta.yaml, and airframe.describe.

• A shared status helper: conditions Ready, Synced plus a closed list of reason codes, each with a hint string, and observedGeneration, on every XRD.
• Verify contracts per component and per app (x-hangar-verify): XR ready, Secret exists, port open, Rollout healthy, SLO not burning.
• airframe.describe: effective config (defaults merged), XR conditions, Argo health, last release, recent failures. Tower’s backend already assembles most of this.
• Accept: every XRD status uses the closed reason list; verify passes and fails correctly on seeded fixtures.
• Effort: 1.5 weeks.

AF-7 Docs for agents (39 → A+)

• Convert the quickstarts to executable walkthroughs (docs/walkthroughs/*.yaml): each step has a command, an expected result and a verify. Render them to markdown; the walkthrough runner replays them on the dev cluster.
• CI extracts every YAML block from the docs and validates it against the contract.
• Reference docs generated from the contract.
• Accept: part 1 replays green; doc YAML is validated in CI.
• Effort: 1 week; rides along with each Skyport part.

AF-8 Interaction surface (27 → A+)

• AppSpec (airframe/contract/appspec.schema.json, vendored byte-identical into autopilot): desired state for an app, its environments and its chart values in one place.
• Planner (autopilot/src/clearance/airframe_plan.py, Built and tested): AppSpec to an ordered ChangeSet across the tenants, app and gitops repos, with gates, dependencies, assumptions and warnings.
• Tools in Clearance: airframe.capabilities, describe, explain, plan, apply (opens PRs), status, verify, diagnose (via Holmes).
• Idempotent (same spec, same change set, a no-op when converged) and resumable (state on the task_id, PR labels and the audit log).
• Accept: the parachute sentence passes as a Preflight case, plus a resume test.
• Effort: 3 weeks, mostly adapters (GitHub, ArgoCD, Kubernetes).

AF-9 Safety integration (20 → A+)

• Risk classes map to Clearance tiers: ground writes T1, flight PRs T2, creating flight environments T2.
• Escape hatches denied by default; field-level policy tested with negative cases.
• Accept: every high or critical field has a negative test.
• Effort: 3 days after AF-5.

AF-10 Hygiene and determinism (46 → A+)

• Chart tests (helm-unittest or a script), chart CI, and a compatibility matrix (chart version by values fixtures). The fleet baseline runs on every schema change.
• Chart fixes: no Rollout without an image (the parachute bug), more guards with actionable messages, devCluster validated against the registry, an agent: block (below).
• Accept: chart CI green; scorecard runs in CI and fails on regression.
• Effort: 1 week, and the first thing to do.

5. Chart changes, in one list

1. Guard: render no Rollout, RolloutWatch or workload until rollout.image is set. Fixes the image: ':' bug and lets an agent configure before the first deploy. Then flip Features.rollout_guard.
2. Base layer and release file support in the templates and ApplicationSets.
3. fromComponent wiring in env and secrets entries.
4. agent: block for service agents (below).
5. More guards with actionable messages (11 today; the target is every invariant the docs state in prose).
6. devCluster validated against the cluster registry.

The agent: block (service agents use the application path)

A service agent is an ordinary application, so the chart gives its pods what an AgentRun gets:

agent:
  enabled: true
  definition: passenger-assistant          # name of the AgentDefinition in git
  network: clearance+model                 # narrow-only against the definition
  sandbox: hardened                        # rejected on a cluster with no runtime class

It renders audience-bound projected tokens (clearance, model-proxy), a default-deny egress policy with the two exceptions, the runtime-contract environment, and the hangar.io/agent labels. Same guarantees, one implementation of the contract in two places (function-agentrun and the chart), kept in step by a shared conformance test.

6. AppSpec and the planner

The parachute sentence, as the agent writes it (airframe/contract/examples/parachute.appspec.yaml):

apiVersion: airframe/v1
kind: AppSpec
app: parachute
stack: python
environments:
  ground: [dev, test]
  flight: [{name: staging}, {name: prod}]
config:
  rollout: {strategy: canary, steps: [{setWeight: 100}]}
  env: [{name: URL, value: "http://myendpoint.io"}]

plan compiles it, today’s Airframe, into:

With the A+ features on, ground-rollout disappears and the config is written once into a base layer.

Assumptions and warnings the planner surfaces instead of hiding: the only upper cluster is prod so both flight environments go there; two upper clusters and no choice is a question, not a guess; test collides with the test pipeline stage; a single 100% step promotes immediately.

The parachute sentence is one example, not the scope

An agent can request a new app and configure it in any way the chart allows. AppSpec’s config (every environment) and overrides.<env> (one environment, deep-merged over config, lists replace) are the airframe-application chart’s values: every top-level key in values.schema.json with x-hangar-owner: human (rollout, env, autoscaling, podDisruptionBudget, httpRoute, ingress, networkPolicy, components, slos, jobs, …), with the chart’s own strictness, conditionals, component XRD schemas and x-hangar-risk. Airframe’s tools/gen_appspec_schema.py generates that part of the schema from the chart, and CI fails if it drifts. These are not settable: platform-owned keys (appName, cluster, envName, …), release-owned keys (release, releaseTracking), appType, and the deprecated rollout.image, because images come from releases.

A second sentence holds it to that (airframe/contract/examples/mynewpythonapp.appspec.yaml):

provision a new python app with a dev, test ground envs, staging, pre-prod and prod flight envs, port 1234, replicas 3, 4 canary steps: 25% weight, pause 30s, 50% weight, pause 30s. 100% weight, a pdb enabled, an hpa enabled, a route called https://mynewpythonapp.hangar.io/.

What the planner adds on top of passing values through:

• Per-environment validation. Each environment’s merged values are checked against the chart again, because two valid halves can make an invalid whole.
• A hostname belongs to one environment. The same hostname in more than one environment (for example a route in config) is a question naming the environments, not a guess.
• Ports agree. stackOptions.port sets the chart’s rollout.ports to http:<port> unless config sets ports itself. A list that leaves the port out is rejected; otherwise the Service would miss the scaffolded server.
• Overrides must name a declared environment.
• The HPA owns the count. rollout.replicas above autoscaling.min with the HPA on is reported, not silently “fixed”.
• Critical-risk values gate the merge. A critical-risk key (extraManifests) moves the app-repo step from auto-merge to human merge.
• Verify checks every claim. The verify step gets one check per claim: replicas, ports, HPA, PDB, and the route per environment. An HTTPRoute has no scheme, so “https” in a request depends on a TLS listener on the Gateway; prod’s kiac Gateway has only HTTP :80 today.

The planner tests render every environment of the second sentence on the real chart.

7. Migration and safety

• Never edit shared things in place. Test-via-copy: chart and schema changes go through a scratch app first; a new composition through one XR’s compositionRef.
• Worktrees. Airframe, glidepath and apron have all been on the user’s branches before. Check the branch, work in a worktree, and leave tags to the user.
• Strictness in stages: warn, sweep the fleet (dev and prod), enforce.
• Multi-arch: everything new builds for arm64 and amd64.
• Unverified, check first: how ArgoCD applies XRs (are unknown fields rejected or pruned?), the git-generator exclude, Helm and unknown x- keywords, and whether chart and schema changes need a chart version bump per your release process.

8. Run it

python3 tools/airframe-scorecard/scorecard.py                  # the grade
cd ~/tech/autopilot && ./.venv/bin/python -m pytest -q          # 255 tests incl. the parachute planner