Docs · Tower

Break-glass debugging

Sometimes logs, metrics, events and the resource tree are not enough. You need to look inside a running pod: list its processes, resolve a name from its network, open a connection it holds, read a file it wrote. Break-glass gives you that: a recorded, time-boxed shell in a hardened debug container, opened from Tower, on one pod of your own app.

You never get kubectl exec, and you never hold a Kubernetes credential. Tower’s backend opens the session for you, watches it, records every byte, and cleans up when it ends.

When to use it, and when not to

Every session is audited and its recording is kept for a year. On Flight a second person approves it first. Treat it the way you would treat a page to a colleague: worth it when you need it.

Ground and Flight

What happens when you press Debug depends on the environment’s tier.

How Tower decides the tier (the request never says):

1. A PR preview namespace (labelled hangar.io/ephemeral-env: "true") is Ground.
2. Otherwise the app’s cicd.yaml decides, read on the server: an environment it declares as Ground (in deploy.environments, or the older lowerEnvironments) is Ground, one it declares as Flight is Flight. This is the same tier the Environments tab shows.
3. An environment cicd.yaml does not name (an infra service such as skyport-auth has no cicd.yaml at all) takes its cluster’s tier: the dev cluster is Ground, every other cluster is Flight.

The life of a session

• Requested (Flight): waiting for a second person. Expires after 30 minutes if nobody decides.
• Approved (Flight): you have 10 minutes to press Start. The session’s own clock starts then.
• Active: the debug container is running and the recording is on. It lasts 15, 30 or 60 minutes, plus one free 15-minute extension if you need it.
• Ended: by you, by the clock, after 10 idle minutes, 2 minutes after you close the terminal without reopening it, when the shell exits, or by an admin or the approver. Tower removes the grant and the debug container, restarts the pod if needed, and stores the recording.
• Review due (glass broken only): until someone who could have approved it, or a fallback reviewer, watches the recording and signs it off. Reviewers are reminded after a day and the admins alerted after three.

You can hold one open request or session at a time, across all apps.

Where to find things

• A pod’s Debug panel (Topology, the environment, Pods, a pod, Debug): the form to open a session on that pod, and the environment’s sessions under it.
• The Break-glass card under Pods in Topology: the environment’s sessions without choosing a pod. A live session restarts its pod and a copy is deleted at the end, so this is where a finished session (and its review) lives. Approvals, reviews, play and download are here; a badge counts what needs you.
• The Break-glass strip on the Tower home page and the Ops Wall: across every app, the requests you can decide, your approved request waiting for its Start, and the reviews you can do. It is empty, and hidden, most of the time.
• Notifications link straight to the session they are about.

Guides

Running the platform? The admin guide covers setup, configuration, the admission policies, operations and internals.