Production commands#

Usage:
  zyn production [command]

Commands:
  access             request and administer read/write access
  db                 query and inspect the production database
  jobs               inspect production job queues and worker logs
  logs               read production application logs
  metrics [--watch]  show production host metrics
  status [--watch]   show production service health

Alias: zyn prod

Inspect production, query its database, and manage individual queued jobs without deploying.

Example:
  zyn prod status

Documentation: zyn docs cli/production

zyn prod status                  # Check the serving release and health
zyn prod status --watch          # Watch status and one-hour metrics
zyn prod metrics 1h              # Inspect CPU, memory, and disk usage
zyn prod metrics --watch         # Watch metrics; arrows change the range
zyn prod logs --follow           # Follow application logs
zyn prod jobs logs --follow      # Follow worker logs
zyn prod jobs status             # Inspect queue counts
zyn prod db query 'SELECT now()'    # Run SQL with the available production access

zyn prod is an alias for zyn production. Commands use the SSH host in an existing deploy.zon. Inspection is read-only; job retry and discard change queue state and require confirmation. SQL queries use the available read or write access without a confirmation prompt. Without a saved access request, SSH requires root access or non-interactive passwordless sudo. For automation, add --no-input: the host key must already be verified in known_hosts, and key authentication must work without a prompt.

Read and write access#

Read grants permit status, logs, jobs logs, metrics, and unfiltered jobs status, including watch and follow modes, and read-only query statements. They do not permit deployment, shell commands, service changes, other job operations, access management, application credential files, or database mutations. Logs and query results may contain sensitive application data. Read access still causes operational writes such as connection and audit records. db status is also available with read access.

Write grants provide full root administrative authority, including deployment, infrastructure changes, job operations, and access management. They can change the access controls themselves or establish persistent access. Expiry and revocation cannot undo changes, retract copied secrets, or contain a malicious administrator.

On the agent's machine, from the application directory:

zyn prod access request

This creates an unencrypted Ed25519 private key and public key beneath $HOME/.local/state/zyn/access/HOST_DIGEST/, outside the repository. Protect the agent's account and disk as credentials. Repeating request reuses the key. Send only the public key to the administrator through an authenticated channel and compare its fingerprint. request needs deploy.zon but does not contact production. Existing saved requests and read grants remain valid; legacy local identities are reused without rotation. Conflicting old and new local profiles fail instead of choosing a credential.

On the administrator's machine:

# Read access by default.
zyn prod access grant debug-agent --public-key agent.pub --expires-in 24h
# Explicit full administrative access.
zyn prod access grant release-agent --public-key release.pub --write --expires-in 24h
zyn prod access list --json
zyn prod access revoke debug-agent

Grant names start with a lowercase letter and contain up to 48 lowercase letters, digits, or hyphens. Public keys must be single plain Ed25519 keys, without authorized-key options. Conflicting names or keys are rejected. To change an active grant's mode, revoke it first and reapprove the same name and key; reapproval does not revive old sessions. --expires-in accepts positive integer s, m, h, or d durations up to 365 days; omission means no expiry. list --json returns schema_version: 1, command: "production access list", and grants with name, logical user identifier, fingerprint, mode (read or write), expires_at (Unix seconds or null), and active. Legacy grants without a mode are read grants.

Provisioning and transport#

zyn deploy provisions access after activating the application, including on existing installations. It installs a native Zig helper, refreshes the fixed inspection scripts and database identities, and reports access readiness separately from application deployment. An access failure does not roll back a healthy application. Correct the reported problem and redeploy; grant also repairs provisioning before approving a key. Inspection never provisions the host.

By default Zyn reuses the administrator's directly reachable OpenSSH listener when its policy supports restricted root keys. Otherwise, with connected Tailscale, it creates zyn-access.service: ordinary OpenSSH bound only to the host's Tailscale IPv4 address, on port 2222. Tailscale SSH and provider SSH keep port 22; their configuration is unchanged. Both administrator and agent must have network access to the private port, permitted by the tailnet's network policy. Tailscale SSH policy does not authenticate this listener.

The managed listener accepts only forced-command grant keys, not administrator passwords, SSH certificates, forwarding, or SFTP. Grants are logical identities, not Unix login accounts. Each key selects its own grant inside the root-owned helper; authenticating through Tailscale SSH cannot select that grant or bypass its restrictions. Root access granted independently by a provider or tailnet policy remains administrative access.

Deployment saves access_endpoint in deploy.zon, containing the endpoint and its Ed25519 host key obtained through administrator SSH. Commit deploy.zon to a repository trusted by your agents, and review host-key changes before accepting them. The record contains schema_version, deployment, host, port, host_key, and login; it contains no private keys or access grants. Private agent keys remain under ~/.local/state/zyn/access/<deployment-hash>/. Saved requests use the shared metadata to pin the server in a private known-hosts file. They ignore user SSH configuration and disable agent identities, connection sharing, and forwarding. A missing, rejected, expired, or revoked saved credential fails without trying administrator credentials. Without a saved request, normal deployment SSH is unchanged. SSH aliases and ProxyJump are unsupported for access-key connections.

Before an older deployment has been upgraded, saved requests continue using its original host on port 22, with the SSH user's verified known-hosts entry. After upgrading, agents need the administrator's updated deploy.zon; they keep their existing private keys. The first migration must run through the original administrator connection, not the legacy grant account being retired.

Existing deploy.access.json files remain readable until the next successful access provisioning through deployment or grant. Provisioning writes the verified metadata into deploy.zon, preserving unrelated configuration and comments, then removes the legacy file. If both records exist, agent connections require them to match. Invalid metadata fails without falling back to administrator credentials.

Migration preserves grant names, keys, modes, generations, and expiry, without reactivating expired grants. It removes verified legacy grant Unix accounts and sudo rules so Tailscale SSH cannot turn them into alternative login paths. Re-running deployment preserves existing grants and the managed host key. Unmanaged files, ambiguous SSH policies, occupied ports, conflicting endpoint overrides, and unreachable endpoints fail access provisioning rather than changing provider listeners, tailnet policy, or firewalls.

The host requires OpenSSH (/usr/sbin/sshd), Linux pidfd support, and systemd. Tailscale must already be connected in TUN mode for a managed private listener; Zyn does not install or enroll it. Install missing OpenSSH server binaries without replacing the provider's listener, then redeploy. See Deployment for optional deploy.zon.access overrides.

The helper enforces the grant mode. Read requests execute only fixed installed inspection scripts under a fixed environment. Write requests may execute administrative shell commands and the existing deployment scripts. At most four supervised sessions per grant run concurrently. Non-streaming inspections have a 30-second execution limit; write commands are not subject to that inspection limit. Log requests accept at most 10,000 initial records.

Expiry and revocation are checked every 200 ms during supervised operations and stop their foreground process groups. Revocation removes the grant, authorized key, and any verified legacy account, home directory, and sudo rule, and signals active supervisors. Revoked grants no longer appear in access list. Access requires a new administrator grant; a new access request can supply its key. If cleanup fails, access remains disabled; retry revocation to finish removing the grant. Revocation must use a different write identity or the deployment administrator; revoking the identity running the command is rejected without changing its grant. An interrupted deployment may already have changed production: inspect zyn prod status before retrying; interruption is not a rollback guarantee. Administrative changes and access attempts are audited through authpriv syslog under zyn-access, without command contents or results.

Database#

zyn prod db connect
zyn prod db status
zyn prod db status --json

db connect opens bundled psql against the active release's application database. It requires a terminal and administrator or write access. The console uses the restricted write database and Unix identities; client shell commands never run as root. Read grants cannot open a console. Saved write keys created before terminal access was supported must be revoked and granted again by an administrator. Authentication failures never fall back to another identity.

db status is available with read access. It reports connectivity, applied application migration versions, pending versions from the active deployed release's db/migrations, and applied versions absent from that inventory. It does not compare the local checkout, apply migrations, or provision access. Pending migrations do not make status fail. An unavailable database or failed migration inspection exits 1; SSH, setup, and protocol failures also exit 1.

--json emits schema_version: 1, command: "production db status", connected, sorted applied, pending, and unmatched version arrays, and error. Successful inspection has a null error. Connection or inspection failure has a string error and null arrays; connected distinguishes them. Setup and SSH failures do not emit a status document.

SQL queries#

zyn prod db query 'SELECT id, title FROM articles ORDER BY id LIMIT 10'
zyn prod db query "UPDATE articles SET title = 'Updated' WHERE id = 42"

db query accepts one UTF-8 SQL argument, at most 65,536 bytes without NUL. It executes one PostgreSQL preparable statement: SELECT, VALUES, TABLE, INSERT, UPDATE, DELETE, or MERGE, including WITH clauses. SQL batches, transaction control, client meta-commands, and schema/administrative commands are not supported. Supply literal SQL values; this command has no parameter-binding interface. SQL travels over SSH standard input, not through remote shell interpolation.

The host chooses the database identity from the authenticated grant. Read grants use a separate non-administrator login with data-reading privileges and an explicit read-only transaction. PostgreSQL rejects writes, including writes inside functions and data-modifying CTEs; supplied SQL cannot turn the transaction writable or assume the application role. Row-level security remains applicable. Reader grants cover non-system schemas' tables and sequences, not protected catalogs such as password verifiers. Future application-role-owned schemas, tables, and sequences inherit reader access. Objects subsequently created by other owners require a deployment or administrator access grant to refresh reader privileges. Read queries conservatively refuse databases containing any SECURITY DEFINER routine owned by a privileged role, even when direct execution is revoked. An administrator must remove the definer attribute, assign a restricted owner, or remove the routine. Application-role definer routines remain permitted. Installed database code and extensions remain trusted: read-only access does not guarantee the absence of external or operational side effects.

Write grants and existing root/passwordless-sudo access use a separate non-administrator login inheriting the application database role's privileges. Successful statements commit; execution failures roll back. Neither mode authenticates to PostgreSQL as its administrator. Queries target only the active release's application database and do not load application credentials, build, deploy, start services, or provision access. Missing or rejected saved access never falls back to administrator credentials.

Row results are streamed as UTF-8 CSV with a header, including for empty result sets. Non-null values are quoted PostgreSQL text representations; unquoted \N represents SQL NULL. Statements without result columns print their PostgreSQL command tag, such as UPDATE 1. Diagnostics go to stderr. Exit status is 0 on success, 1 on a remote/database failure, or 64 for invalid CLI arguments. Output may be partial on failure; only exit status 0 confirms a write committed. Each statement has a 25-second execution timeout, a two-second lock timeout, and a 64 MiB output limit. Exceeding the output limit aborts the transaction. Queries retain access-session concurrency, expiry, revocation, and audit rules.

Status#

zyn prod status [--verbose] [--json | --watch] [--no-input]

See a terminal status example.

Reports the serving release and slot, HTTP healthcheck, service states, uptimes, deployment and maintenance state, certificate expiry, and one hour of CPU, memory, and disk history. Add --verbose for inactive slots and component details.

Healthy status exits 0; degraded or unknown status exits 1. SSH or response failures also exit 1; invalid command syntax exits 64. --json includes the full snapshot, even without --verbose.

--watch refreshes in place and requires terminal output. It reconnects after connection failures and cannot be combined with --json. Press Ctrl-C to stop.

Metrics#

zyn prod metrics [DURATION] [--watch] [--no-input]

See a terminal metrics example.

Shows CPU, memory, and filesystem usage over the last 24 hours by default. Durations range from 1m to 1y, using m, h, d, w, or y. Each chart includes latest-bucket used/total capacity: vCPUs for CPU and binary units for memory and disk. CPU is a bucket average; memory and disk are the last observation. Older history remains readable, with -- for unrecorded capacities. Terminal output shows loading chart outlines while the report is collected.

--watch defaults to five minutes and requires terminal output. With terminal input, left and right arrows select 1m, 5m, 15m, 1h, 24h, 7d, 30d, or 1y; from a custom duration they select the nearest shorter or longer range. It reconnects after connection failures. Press Ctrl-C to stop.

Missing intervals stay blank; unavailable or stale latest values show --. History starts when the collector is installed: minute samples are retained for 90 days and hourly summaries for one year. Short ranges require the recent collector installed by zyn deploy.

Logs#

zyn prod logs [--all] [--follow] [--lines COUNT] [--no-input]

Reads the newest 20 application records across both deployment slots in journal order. Use zyn prod jobs logs for worker records. --lines COUNT sets a non-negative limit; --follow streams through later deployments; --all includes routine health and shutdown records. Gateway, PostgreSQL, and systemd lifecycle records are excluded.

Timestamps are UTC. Request and job IDs are shortened to eight hexadecimal characters; full IDs remain in the remote journal. Full request IDs also appear in the X-Request-ID response header.

Jobs#

zyn prod jobs status --details   # Counts and eligibility
zyn prod jobs failed --limit 20  # Retained failures, newest first
zyn prod jobs show ID            # Inspect one retained job
zyn prod jobs retry ID           # Retry failed or discarded work
zyn prod jobs discard ID         # Discard pending or scheduled work

Status and failure listings accept --queue NAME; read-only commands accept --json. Inspection excludes arguments and credentials. Commands use the active release's managed database; zyn jobs targets local development.

Retry resets attempts and makes a job immediately eligible. Discard does not cancel running handlers. Both require confirmation by typing yes, or explicit --yes; automation requires --yes --no-input. They refuse active deployments, maintenance, and pending activation. If a connection fails, inspect the job before retrying: the change may already have committed.

See Production monitoring for health thresholds, metrics collection details, and JSON field definitions.