zynagent-first application framework

Storage#

Zyn stores immutable blobs through storage on actions, hooks, middleware, jobs, and managed test applications. Disk and S3-compatible profiles expose the same API. Applications store blob identifiers and own their attachment tables.

Write and read blobs#

write stores bytes and metadata and returns a zyn.Blob:

const blob = try action.storage.write(contents, .{
    .filename = "summary.csv",
    .media_type = "text/csv; charset=utf-8",
});

A blob contains id, filename, media_type, byte_size, sha256, metadata, and a Unix-microsecond created_at. Returned strings and metadata live for the current request, job, or managed-test application scope. Zyn generates its backend key; application code cannot select or read it. Blob bytes are immutable. Replacing content creates a new blob and the application updates its reference.

expected_sha256 optionally rejects bytes whose SHA-256 digest differs. Zyn always computes and records the digest. Filenames must be non-empty UTF-8 without control characters, and media types must be syntactically valid.

get returns metadata by identifier. read returns null when the blob was purged and allocates from the current request, job, or test scope. Its std.Io.Limit bounds the allocation and a complete buffered read verifies the stored digest:

const blob = (try action.storage.get(blob_id)) orelse return error.NotFound;
const contents = (try action.storage.read(blob, .limited(1024 * 1024))) orelse
    return error.NotFound;

Framework-derived media properties live in Blob.metadata. Image dimensions, when present, are metadata.image.width and metadata.image.height. Application-specific metadata belongs in application tables.

Image variants#

Generate named image derivatives in background jobs.

Define variants#

Export your storage definitions through src/app.zig:

pub const storage = @import("storage.zig");

Declare named variants in src/storage.zig:

const zyn = @import("zyn");

pub const variants = .{
    .avatar = zyn.ImageVariant{
        .resize = .{ .cover = .{ .width = 256, .height = 256 } },
        .format = .webp,
        .quality = 80,
    },
    .preview = zyn.ImageVariant{
        .resize = .{ .contain = .{ .width = 1200, .height = 1200 } },
        .format = .jpeg,
        .quality = 85,
    },
};

Names are checked during compilation. Variant definitions are not supplied through URLs. resize is optional: omission retains oriented dimensions. contain preserves proportions within the bounds, without padding; cover center-crops to exact width and height. Dimensions must be 1–16384. upscale defaults to false; cover fails if the source cannot fill the requested size without enlargement. format must be jpeg, png, or webp. JPEG/WebP quality defaults to 80 and accepts 1–100; specifying it for PNG is a compilation error. JPEG flattens transparency against background, an RGB byte array defaulting to .{ 255, 255, 255 }. PNG and WebP retain transparency.

Request variants when writing an original:

const original = try action.storage.write(contents, .{
    .filename = "portrait.jpg",
    .media_type = "image/jpeg",
    .variants = &.{ .avatar, .preview },
});

writer accepts the same variants option. finish publishes original metadata and the generation jobs in one database transaction after publishing bytes. An unfinished or rejected write queues nothing. The returned blob contains the unchanged original bytes and metadata. Upload success means stored and queued, not validated as an image or processed. Existing attachment insertion remains an application-owned transaction, separate from uploading.

For an existing original, queue work by its ID:

try action.storage.generate_variants(original.id, &.{.avatar});
try tx.storage.generate_variants(original.id, &.{ .avatar, .preview });

Transaction-bound requests become visible on commit. Ready results are reused, pending work is not duplicated, and absent or terminally failed work is queued. Repeated names and equivalent normalized variant definitions share one result. Generation uses the background-job retry and lease machinery. A request does not wait for processing; successful completion means the request was recorded, subject to commit for transaction-bound calls.

Lookup is read-only and accepts the original ID without first loading its blob:

const result = try action.storage.variant(original_id, .avatar);
return switch (result) {
    .ready => |blob| action.storage.serve(blob, .{
        .disposition = .inline_content,
    }),
    .absent => .empty(.{ .status = .not_found }),
    .pending => .empty(.{ .status = .accepted }),
    .failed => error.VariantGenerationFailed,
};

The application authorizes access before lookup and chooses pending/error responses. zyn.Variant is absent, pending, ready: zyn.Blob, or failed: zyn.VariantFailure. Missing or purged originals return error.BlobNotFound; absent means that variant version was not requested. pending includes automatic retries. Failure categories are invalid_image, unsupported_format, limits_exceeded, insufficient_resolution, and processing_failed; terminal failure survives job-history pruning. No raw processor diagnostics are exposed through this result.

A ready variant supports existing serve, read, reader, public_url, and temporary_url methods. Delivery does not start processing or substitute the original. Purging an original removes every derivative version and schedules backend deletion in the same transaction. Purging a derivative directly removes its lookup entry; another generation request can recreate it. Application foreign keys continue to protect attached blobs, including derivatives. Concurrent or superseded jobs cannot republish purged derivatives.

Identity includes the original ID, normalized variant definition and processor revision. Renaming an unchanged variant reuses its result. Changed definitions require explicit generation for existing originals; configuration changes do not start backfills. Queued jobs retain a snapshot of the variant definition. Deployment rejects queued payloads for an incompatible processor revision; drain them before upgrading that revision.

Image processing and privacy#

The processor decodes JPEG, PNG and WebP by content, not declared media type. Outputs are static; animated inputs use the first frame. Processing applies EXIF orientation, converts color to sRGB, and strips source metadata including EXIF, GPS, XMP, IPTC and embedded profiles. Image dimensions on a ready blob describe its output pixels.

Originals retain all uploaded metadata, including location. Public storage URLs can expose originals independently of application routes. Use private storage and authorized delivery when originals must not be public; there is no per-blob visibility override.

Resource ceilings belong in zyn.zon, outside backend profiles:

.storage = .{
    .images = .{
        .max_input_bytes = 20_000_000,
        .max_pixels = 40_000_000,
        .max_output_bytes = 40_000_000,
        .timeout_seconds = 30,
        .concurrency = 2,
    },
    // profiles...
},

These are the defaults; every value must be positive. max_pixels bounds source and output dimensions. concurrency bounds processor subprocesses sharing the worker's spool directory, in addition to the general job-worker limit. Existing storage read and write size ceilings also apply. Processing runs in a separate helper with time and file-size limits and, on Linux, an address-space limit. The subprocess boundary is not an authorization sandbox.

The helper supports x86-64/AArch64 Linux with glibc 2.28+ and supported macOS hosts. A musl application binary still uses the separate GNU helper on Linux. Development/testing provision the checksummed runtime on first processing; managed deployment provisions and smoke-tests it before activation. Initial provisioning requires outbound HTTPS, curl, tar, and, on Linux, xz and GNU dd. See Deployment for shared runtime installation.

Delivery and URLs#

A response-scoped storage capability can stream a blob:

return action.storage.serve(blob, .{});
return action.storage.serve(blob, .{ .disposition = .inline_content });

Delivery defaults to attachment. It sets the recorded media type, an encoded content disposition, Content-Length, X-Content-Type-Options: nosniff, a strong SHA-256 entity tag, and Last-Modified; it applies request preconditions and suppresses bytes for HEAD.

Configured public and time-limited URLs identify immutable blob bytes:

const public = try action.storage.public_url(blob, .{});
const temporary = try action.storage.temporary_url(
    blob,
    .fromSeconds(15 * 60),
    .{},
);

public_url fails when public delivery is not configured. Temporary lifetimes must be positive and may not exceed storage.max_temporary_url_seconds. Disk delivery checks that blob metadata still exists before serving bytes. S3-compatible writes record attachment disposition and media type as object headers.

Template helpers accept blob identifier strings and provide URL generation only:

<img src="<%= storage_url(@avatar_blob_id) %>" alt="Avatar">
<a href="<%= temporary_storage_url(@document_blob_id, 900) %>">Download</a>

Zig I/O#

writer and reader avoid buffering complete contents in application memory:

var output = try job.storage.writer(.{
    .filename = "report.pdf",
    .media_type = "application/pdf",
    .expected_size = report_size,
});
defer output.deinit();
try generate_report(&output.interface);
const blob = try output.finish();

var input = (try action.storage.reader(blob)) orelse return error.NotFound;
defer input.deinit();
try input.interface.stream(&destination, .unlimited);

Writer.finish verifies the exact declared size and optional digest before publishing metadata. Writes that exceed the declared size fail before excess bytes reach a backend, and a write failure poisons the writer. Calling deinit without finishing discards the pending write. Readers never observe partial writes.

S3-compatible writers spool to a private local file and do not begin the final object PUT until finish has validated the complete input. spool_root must be on a writable local filesystem with enough capacity for concurrent pending writes. A failure after an object upload begins but before its success response is received returns error.StoragePublicationUncertain; operators must check that backend before retrying application work that is not otherwise idempotent.

Backend publication and PostgreSQL metadata cannot share one transaction. Zyn publishes bytes, inserts and fully decodes metadata inside a PostgreSQL transaction, and then commits. Failures before commit attempt to remove the published bytes; failed compensation can leave orphaned backend bytes. A commit error returns error.StorageMetadataCommitUncertain and retains the bytes because PostgreSQL may have committed the row. Process termination or uncertain publication can leave backend bytes without a blob row, but cannot expose an incomplete blob row.

References and purging#

Application migrations may reference the framework-owned blob table. Generate the application table:

zyn g table post_images

Add the reference columns and pairwise uniqueness constraint, retaining the generated UUID primary key, timestamps, and update trigger. For a posts table with UUID IDs created by zyn g table posts, the table declaration becomes:

CREATE TABLE public."post_images" (
    id uuid NOT NULL DEFAULT uuidv7() PRIMARY KEY,
    created_at timestamptz NOT NULL DEFAULT now(),
    updated_at timestamptz NOT NULL DEFAULT now(),
    post_id uuid NOT NULL REFERENCES posts(id),
    blob_id uuid NOT NULL REFERENCES zyn.blobs(id),
    UNIQUE (post_id, blob_id)
);

Run zyn db migrate after editing the migration.

Zyn does not create polymorphic attachment rows or infer ownership.

purge deletes blob metadata and enqueues the reserved zyn.storage.purge job in one PostgreSQL transaction. Application foreign keys reject purging an attached blob. The job deletes backend bytes idempotently with the configured job retry and lease machinery:

var tx = try action.db.begin();
defer tx.cancel();
try tx.exec("DELETE FROM post_images WHERE blob_id = $1::uuid", .{blob.id});
try tx.storage.purge(blob);
try tx.commit();

action.storage.purge(blob) performs the same operation in its own transaction. Deployments and development start a job worker when either application jobs or configured storage require one.

Database schema#

Zyn owns zyn.blobs, zyn.variants, and its schema revision bookkeeping. The table stores the blob UUID, service, private backend key, filename, media type, byte size, SHA-256 digest, framework metadata JSON object, and creation time. db/zyn_schema.sql contains the generated definition and is excluded from db/types.zig and the application schema dump. Setup, migration, reset, deployment, and managed testing install the current framework definition before restoring application tables so application foreign keys can reference blobs.

Configuration#

Storage is optional. Define profiles in zyn.zon:

.storage = .{
    .max_entry_size = 1_073_741_824,
    .max_read_size = 1_073_741_824,
    .max_temporary_url_seconds = 604_800,
    .profiles = .{
        .development = .{
            .service = "development_files",
            .disk = .{ .root = ".zyn/storage", .public = true },
        },
        .testing = .{
            .service = "testing_files",
            .disk = .{ .temporary = true, .public = true },
        },
        .production = .{
            .service = "production_files",
            .s3 = .{
                .endpoint = "https://objects.example.com",
                .bucket = "application-files",
                .region = "us-east-1",
                .prefix = "production/",
                .addressing = .path,
                .public_base_url = "https://files.example.com",
            },
        },
    },
}

An absent active profile leaves storage unconfigured. All blob operations, including metadata lookup and transaction-bound purge, then return error.StorageNotConfigured.

Each profile may define a stable service name. Blob rows and purge jobs retain that logical identity, preventing a process configured for another service from reading or deleting its bytes. Omission retains the legacy backend names disk and s3. Pin an explicit service before relying on location isolation. Changing a service name requires migrating zyn.blobs.service and pending zyn.storage.purge job payloads together with the bytes. Changing a selected backend or storage location without retaining the same logical service requires the same migration.

max_entry_size bounds new writes. max_read_size independently bounds opened readers and response delivery, including existing objects; read also applies its supplied std.Io.Limit to the allocation.

The disk backend stores entries beneath its root using backend-owned names and atomic publication. It synchronizes file contents and the object and shard directories on platforms with directory synchronization support. .temporary = true is accepted only for managed testing. .public = true enables the reserved /_zyn/storage/ delivery route.

The S3-compatible backend uses Signature Version 4 and supports object PUT, GET, HEAD, and direct idempotent DELETE. It spools pending writes beneath an explicit spool_root, a managed deployment runtime directory, or .zyn/storage-spool otherwise, before sending the final PUT. HTTPS is required except for loopback endpoints. It does not administer buckets, ACLs, policies, identities, lifecycle rules, or provider extensions. public_base_url must be an HTTP(S) URL without credentials, query, or fragment and identifies the public root corresponding to the configured prefix.

S3 credentials use optional private credential fields at storage.s3.access_key_id, storage.s3.secret_access_key, and storage.s3.session_token, unless the profile overrides their paths.