Deployment#

Deploy one application to one x86_64 or AArch64 Linux host over SSH. Zyn manages PostgreSQL, systemd services, and blue/green releases. Domains enable automatic HTTPS; without a domain, deployment uses HTTP.

First deployment#

From the application directory:

zyn deploy

The first run asks for an SSH host, an optional domain, and, without a domain, an HTTP port (default 80). It writes deploy.zon, prepares encrypted production credentials, and runs application tests. --skip-tests skips the tests.

The host needs:

• x86_64 or AArch64 Linux with glibc 2.31+, systemd, flock, tar, and sha256sum;
• root SSH access or non-interactive passwordless sudo;
• apt-get if curl must be installed;
• for HTTPS, domains resolving to the host and public ports 80 and 443;
• outbound HTTPS for the pinned Caddy download unless compatible Caddy 2 is installed.

Zyn checks root access before building, records unseen SSH host keys, and rejects changed keys. It uses normal OpenSSH configuration but disables terminal allocation for binary uploads, including with RequestTTY force. Unmanaged service or database layouts, unsafe/incompatible Caddy executables, and an active unmanaged caddy.service are rejected. For automatic HTTPS, Caddy owns ports 80 and 443; certificates persist between releases.

Configuration#

.{
    .hosts = &.{ "203.0.113.10" },
    .domains = &.{ "app.example.com" },
    .tls = .automatic,
    .healthcheck = .{ .path = "/healthz" },
}

Configurations without domains or tls remain HTTP deployments.

Production access#

Deployment also provisions restricted production access. It reuses compatible administrator OpenSSH, or runs a separate OpenSSH listener on a connected Tailscale IPv4 address at port 2222. Neither Tailscale SSH nor a provider's SSH listener is modified. The access helper and SQL runner are compiled Zig executables; no Python runtime is required on the deployment host.

For a VM using Tailscale SSH or provider-managed SSH, defaults need no additional application configuration. Optionally select the private listener explicitly:

.access = .{
    .listener = .tailscale,
    // .host = "node.example.ts.net",
    // .port = 2223,
},

Overrides select the private listener and cannot accompany .existing. Provisioned endpoints are stable: conflicting overrides fail rather than silently moving granted identities to another endpoint. No firewall or tailnet-policy rules are added. Agents need a route and permission to reach the chosen TCP port; an HTTP-only VM proxy does not expose it as raw SSH.

After activation, Zyn verifies endpoint reachability against the host key obtained over administrator SSH and saves the endpoint and public host key in deploy.zon.access_endpoint. Commit this metadata to a repository trusted by your agents; review host-key changes before accepting them. Private agent keys remain in each user's local state directory. An access warning means the application was deployed but access is not ready; inspect zyn-access.service through the original administrator connection, correct connectivity or policy, and redeploy. Existing installations upgrade on deployment without reinitializing application data or requiring agents to generate new keys.

See Production commands for requests, grants, migration, and revocation.

Releases and rollback#

Run zyn deploy again for later releases. Zyn builds and uploads a ReleaseSafe application for the host architecture, applies migrations, checks queued-job compatibility, and starts the inactive HTTP slot. Activation and rollback are serialized per host. Ordinary releases do not restart PostgreSQL or rewrite development schema or generated types.

Migrations must work with both releases: the old release serves traffic while they run. Destructive changes and database locks can interrupt requests. Deployment tracks applied versions, not file checksums; preserve migration files for replay on new databases. The former checksum column is removed on the next migration.

Activation requires a private healthcheck 2xx, followed by a gateway check after switching. HTTPS checks verify certificates and SNI through local Caddy and public DNS for every domain. Redirects and transport failures do not pass. Each probe uses a separate, closed connection.

Existing requests and upgraded streams stay on the old release for up to 60 seconds. A drain timeout fails deployment and switches traffic back, preserving the old service and its streams. Rollback allows candidate streams up to 60 seconds before closing them with a warning. Failures report probe results and bounded service journals. The previous release is retained when available. An activation journal restores the prior socket, release, gateway, and Caddy configuration before the next deployment after interruption or reboot.

Ordinary releases do not restart the gateway or reload Caddy. An older gateway's first upgrade to draining support does restart it and interrupts connections. Request.raw.address is the verified client address, not an untrusted forwarding header.

Background workers#

Registered jobs use zyn-blue-jobs.service and zyn-green-jobs.service, matched to HTTP slots and immutable releases. Before activation, the candidate checks:

• production database access, framework schema, and recurring definitions;
• durable names and JSON arguments of pending, leased, and retrying jobs;
• stored concurrency-key digests and limits against candidate policy.

Checks do not claim jobs or change the queue or schedule state. An incompatible handler, payload, or concurrency policy blocks deployment while affected rows remain. Checks cannot prove semantic compatibility or cover jobs created later. Use versioned names such as send_receipt_v2 for incompatible changes and retain old handlers until their work drains, including through the rollback window.

Worker handoff proceeds in this order:

1. Pass the candidate HTTP healthcheck and stop the old worker.
2. Switch traffic, drain and stop old HTTP, then repeat queue compatibility checks.
3. Start the candidate worker, synchronize recurring definitions, and wait for queue access and execution-thread readiness before enabling it.

If draining fails, the candidate worker never starts. During rollback, the old worker restarts only after candidate HTTP and worker services stop. A failed stop leaves the old worker stopped until journal recovery; failed recovery leaves the old HTTP slot serving without its worker. Removing runtime authorization alone does not stop a running worker.

Worker shutdown is bounded to fifteen seconds. Terminated attempts retain their lease until expiry and may run again. Jobs remain durable during handoff. New or changed recurring definitions start at their next future occurrence; removed definitions remain stored but inert, so rollback can resume their cursors. Restored workers synchronize definitions before claiming work.

The activation journal records both worker units and their active/enabled states. Candidate authorization expires on reboot, preventing an uncommitted worker from claiming work before recovery. Releases referenced by HTTP or worker units are retained.

Maintenance and recovery#

zyn deploy --maintenance

Use maintenance for incompatible migrations and planned cutovers. Existing workers on framework schema revision 9 or 10 require maintenance to upgrade; they cannot restart against the new schema.

Tests, build, and upload finish before the outage. The gateway then returns 503 Service Unavailable, Retry-After: 60, and Cache-Control: no-store, independently of application/database health. Both HTTP slots and workers stop before migrations or PostgreSQL binary replacement. HTTP draining waits without a deadline; workers retain their fifteen-second bound.

Traffic reopens only after private healthcheck 2xx, queue compatibility, and required worker readiness. Gateway probes expect maintenance 503; HTTPS probes still verify certificates and every domain. Queued work is not discarded or converted. Older gateways refuse new connections while draining and adopting maintenance responses.

Failure after maintenance begins leaves traffic closed, even after reboot. Zyn does not restore old code against a changed database or reverse database changes. Inspect logs, fix the cause, and retry:

zyn prod logs
zyn prod jobs logs
zyn deploy --maintenance

Retries stop surviving slot processes before continuing. Ordinary deployment cannot recover maintenance, including interruption between activation and reopening. Keep a database backup and recovery plan for incompatible changes.

Infrastructure changes also need a maintenance plan:

• application port, TLS, or domains (edge changes may interrupt connections);
• PostgreSQL major version or port;
• database username or peer/password authentication mode.

Maintenance supports application-port changes and PostgreSQL binary replacement within a major version. It does not permit database port, username, or authentication-mode changes. Major upgrades require operator-managed pg_upgrade or dump/restore. For SCRAM password rotation, change the host database role's password before deploying its new credential.

Assets, storage, and images#

Production assets are minified and pre-encoded. Public source maps participate in the manifest and retention; private .zyn/source-maps files are not uploaded. Identity and encoded representations are independently content-addressed.

Zyn validates the asset manifest and publishes fingerprints to shared storage before candidate startup; matching empty manifests and directories are valid. Old and new assets remain available through activation, draining, failure, and rollback. Pruning follows successful draining and preserves assets referenced by retained releases.

Disk storage needs a shared writable root outside release directories, accessible to both HTTP slots and workers. Deployment does not replicate, back up, or migrate blobs; S3-compatible storage is external to publication. See Storage.

Image variants use a release-local helper and checksummed libvips runtime under /opt/zyn/images/PROCESSOR_REVISION/TARGET. Provisioning requires glibc 2.28+ (even for musl applications), outbound HTTPS, curl, tar, xz, and GNU dd. Runtime installation and a helper smoke test must succeed before activation. Runtimes, including C++ libraries, are retained for rollback, not uploaded with ordinary releases. Work files use the worker's writable spool; no daemon is installed. See Storage.

Production credentials#

Deployment requires db.profiles.production, a database name, and effective host 127.0.0.1; it never falls back to development. Passwordless access requires user zyn over a private peer-authenticated Unix socket. An explicit password enables loopback SCRAM and another application username; zyn-postgres is never accepted. The project's schema defines required credential values.

Credentials are private and isolated per release. Provisioning, migrations, and startup read immutable snapshots without credential-recovery writes; the application identity needs no release-directory write access. Temporary credential transfers and upload archives are removed after success or failure; connection failures may leave owner-only archives. Caddy, the gateway, application slots, and PostgreSQL use separate restricted identities.

Inspect production#

zyn prod status
zyn prod logs --follow
zyn prod jobs logs --follow
zyn prod metrics --watch

See Production commands for options and read/write access, and Production monitoring for health and metrics. Inspection does not deploy or recover releases. Metrics persist between releases; collector setup failures warn without rolling back serving traffic.