zynagent-first application framework

Background jobs#

Zyn stores background jobs in PostgreSQL and runs them with a separate worker process. Register callbacks in src/jobs.zig:

const zyn = @import("zyn");

pub const defaults: zyn.JobDefaults = .{
    .queue = "default",
    .retry = .{ .max_attempts = 5 },
};

pub const registry: zyn.JobRegistry = &.{
    .job("send_receipt", send_receipt, .{ .queue = "mailers" }),
};

fn send_receipt(
    job: *zyn.Job,
    args: struct { order_id: []const u8 },
) !void {
    job.logger.info("sending receipt for {s}", .{args.order_id});
}

The first argument is the durable job name stored with queued work. Renaming or moving the callback does not change that name. Changing a registered name leaves previously queued work under the old name; keep that registration until it drains or migrate the work. For incompatible argument changes, register a versioned name such as send_receipt_v2 and move producers to it. Callbacks have the signature fn (*zyn.Job, Args) !void. Registries may use imported callbacks and compile-time composition.

Concurrency limits#

Limit live leased attempts per job, optionally grouped by an argument field:

.job("sync_account", sync_account, .{
    .concurrency = .{ .limit = 1, .by = "account_id" },
}),

by must name a top-level, non-optional byte-string, integer, or enum argument field; the field and positive limit are checked at compile time. Jobs with the same field value share capacity. Omit by for one limit across all arguments. Queued jobs, including transactional and recurring work, store a SHA-256 key digest and the limit. Arguments remain stored as JSONB.

Only live leases occupy slots. Retrying, scheduled, and pending jobs do not. Completion, failure, discard, or lease expiry releases capacity. Claims use transaction-scoped PostgreSQL locks across worker processes. A handler can continue after losing its lease, under the at-least-once delivery contract. Adding, removing, or changing a policy requires outstanding work to drain or a versioned job name.

Recurring jobs#

Applications may also declare recurring schedules in the same module:

pub const recurring: zyn.RecurringJobRegistry = &.{
    .cron(
        "nightly_cleanup",
        "15 0 3 * * *",
        "cleanup",
        .{},
    ),
};

Each .cron(NAME, EXPRESSION, JOB, ARGS) entry needs a unique strict-snake-case name. Schedule names, expressions, and target job names are compile-time strings. The target must be registered; arguments follow the same validation and payload limit as enqueue.

Cron expressions use UTC fields for second, minute, hour, day of month, month, and day of week. Five-field expressions imply second 0. Fields accept *, lists, ranges, and / steps. Seconds and minutes range from 0 to 59, hours from 0 to 23, days of month from 1 to 31, months from 1 to 12, and days of week from 0 to 7 (0 and 7 are Sunday). If neither day field starts with *, either may match; otherwise both must match. Timezones, macros, and interval schedules are not supported. Expressions that cannot produce a date are rejected at compile time.

New or changed schedules start at their next future occurrence. An unchanged schedule that missed runs enqueues only the latest missed occurrence. Removing a schedule stops it; restoring it resumes from its previous position. Recurring jobs use the registered job's queue and retry policy.

Enqueueing jobs#

Actions, hooks, middleware, and jobs enqueue registered work through jobs. Without scheduling options, a job is ready as soon as the insert commits:

try action.jobs.enqueue("send_receipt",
    .{ .order_id = order.id },
    .{},
);

A database transaction exposes the same enqueue contract through tx.jobs:

var tx = try action.db.begin();
defer tx.cancel();

try tx.exec(
    "UPDATE orders SET paid = true WHERE id = $1",
    .{order.id},
);
try tx.jobs.enqueue("send_receipt", .{ .order_id = order.id }, .{});

try tx.commit();

The database changes and job insert commit or roll back together. Transactions from actions and jobs support tx.jobs; enqueueing after commit or cancellation returns error.TransactionCompleted.

Set after for a relative delay or at for an absolute Unix-microsecond timestamp:

try action.jobs.enqueue("send_receipt", .{ .order_id = order.id }, .{
    .after = .fromSeconds(30),
});
try action.jobs.enqueue("send_receipt", .{ .order_id = order.id }, .{
    .at = send_at,
});

after and at cannot be combined (error.ConflictingJobSchedule). Nonpositive delays and past timestamps are immediately eligible. Out-of-range times return error.JobScheduleOutOfRange. Scheduled time is the earliest claim time, not a deadline; priority only orders eligible jobs.

Arguments must match the registered callback's type, serialize to JSON, and fit the configured payload limit. They are stored as PostgreSQL JSONB: do not enqueue secrets.

Running jobs#

A job has database, enqueue, notification, and storage capabilities, plus a logger, I/O, an attempt-scoped arena, its attempt number, and shutdown state. Returning an error retries according to the registered policy. job.retry_in(duration) selects a later retry; job.discard() stops retries. Retrying on the final allowed attempt records a failure.

Jobs use at-least-once delivery. A worker crash, lost lease, or interruption may execute a handler again, so handlers must make external effects idempotent. Completed, failed, and discarded jobs remain recorded until their retention period expires. Queue names do not create separate worker pools; workers claim from one priority-ordered stream. Workers poll even when PostgreSQL notifications are unavailable.

Applications started by zyn.testing.start can run jobs synchronously through app.jobs without a worker process. Delayed retries require enabling delayed execution in the test operation.

Configuration and operations#

Runtime worker configuration lives in zyn.zon:

.jobs = .{
    .concurrency = 4,
    .poll_interval_ms = 500,
    .lease_seconds = 60,
    .max_payload_bytes = 65_536,
    .successful_retention_days = 7,
    .failed_retention_days = 30,
    .discarded_retention_days = 30,
},

Queue data lives in the application's PostgreSQL database under the reserved zyn schema. The generated schema snapshot is db/zyn_schema.sql. Queue operations reject missing, incomplete, outdated, or newer unsupported schemas. Graceful shutdown stops new claims and lets active handlers finish; handlers can check job.stopping(). Lease heartbeats try concurrency advisory locks without waiting and retry a busy lock on a later tick. Heartbeat database commands use a one-second client I/O deadline. Attempt outcome persistence does not wait for a heartbeat to join; heartbeat shutdown cancels its outstanding I/O and joins before freeing attempt memory. A database outage can still cause lease expiry and a later retry, so handlers must tolerate repeated execution. Successful jobs are retained for 7 days by default; failed and discarded jobs for 30 days. Set the corresponding retention value to zero to make rows eligible for the next cleanup pass. Use zyn jobs status, completed, failed, retry, discard, and prune to inspect or manage the queue without exposing arguments.

Deployment checks pending jobs against the new release's registered names and argument types before switching workers. The previous worker stops before the new one starts. zyn prod jobs logs includes job-worker logs.

Queue inspection and cleanup#

zyn jobs manages the development queue; Jobs command lists commands and filters. An already running PostgreSQL server is left running; one started for a queue command is stopped afterward.

Workers load recurring schedules before processing jobs. zyn dev starts the worker with PostgreSQL and HTTP when the registry is non-empty; zyn serve is HTTP-only. Deployment uses a slot-matched worker, validates queued payloads without changing recurring schedules before handoff, and synchronizes source schedules before readiness. See Deployment for compatibility and rollback rules.

Attempt summaries contain job name, queue, attempt limit, outcome, total duration, database duration, and queue-wait duration. Error summaries retain an ERROR label; successful summaries omit a routine INFO label. Production worker records are available through zyn prod jobs logs.

Workers periodically prune one bounded batch and schedule another shortly after a full batch. Manual zyn jobs prune uses the same cleanup: oldest terminal rows first, skipping rows locked by concurrent retry or administrative work.

JSON status#

zyn jobs status --json emits:

{"schema_version":1,"command":"jobs status","queue":null,"counts":{"pending":0,"running":0,"completed":0,"failed":0,"discarded":0},"details":null}

queue is the selected queue string, or null for all queues. Counts are integers. --details replaces null details with an object containing integer ready, scheduled, and expired_leases, plus nullable integer oldest_ready_age_ms and next_eligible_at. Ages use milliseconds; the next eligible timestamp uses Unix epoch microseconds. Missing times are JSON null, not human-output sentinel strings. JSON does not include job arguments or credentials. The same local database startup and shutdown lifecycle applies; its messages are written to stderr.