---
title: Testing in CI
url: "https://native-sdk.dev/docs/testing/ci"
docs_index: /llms.txt
lastUpdated: 2026-10-16
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

`native init --full` and web-frontend scaffolds generate `.github/workflows/ci.yml`. The default scaffold does not include CI. Use the recipes below to add a workflow or adapt the generated one.

## Three tiers of confidence

Native SDK apps test in three tiers, each running in progressively more realistic environments:

1. **Logic tests, everywhere** — `native test -Dplatform=null` runs the app’s `Model`/`Msg`/`update` loop, the markup compiler, and the canvas layout engine on the headless null platform. No window server, no GPU, no OS dependencies: these run on any runner (and in the generated workflow's `test` job).
2. **Xvfb automation smoke, on Linux** — the app builds with `-Dplatform=linux -Dautomation=true`, launches a real GTK window under a virtual X display, and is driven through the file-based [automation](/docs/automation) protocol: wait for readiness, assert on the accessibility snapshot, capture a screenshot. This proves the real render path (window, GPU surface, widget semantics) without GUI hardware.
3. **Real windows, on macOS runners** — GitHub's macOS runners provide a GUI-capable session, so AppKit windows actually open there. The same `native automate` commands drive them; the framework's own CI runs its WebView and GPU smoke suites this way.

Most apps get lasting value from tiers 1 and 2 alone — that is what the generated workflow sets up.

## Deterministic screenshots as goldens

`native automate screenshot <view-label>` renders a `gpu_surface` view through the deterministic CPU reference renderer at scale 1. Two captures of an unchanged scene are byte-identical on the same machine, so screenshot diffs can serve as golden checks: commit a known-good PNG and `cmp` against a fresh capture in CI, or just assert the artifact is non-empty as the generated workflow does. Text measurement can differ across OS versions, so compare goldens captured on the same runner image rather than across platforms.

## Snapshot assertions with `automate assert`

`native automate assert` replaces grep-and-sleep chains with one polling command:

```bash
native automate assert 'gpu_nonblank=true' 'role=button name="Reset"' 'total: 0'
```

Each argument is a regex that must match `snapshot.txt`. The command polls until every pattern matches or the timeout expires, then exits 0 (CI-friendly). On failure it exits non-zero, names each missing pattern, and prints the snapshot tail so the CI log carries the evidence.

- `--timeout-ms <n>` — how long to keep polling (default 30000).
- `--absent` — invert the check: every pattern must be gone from the snapshot. Useful for `'error event='` style regressions or waiting for a widget to disappear.

The supported regex subset is grep-like: literals, `.`, postfix `*` `+` `?`, line anchors `^` and `$`, character classes `[a-z]` / `[^0-9]`, and `\d` `\w` `\s` escapes (no groups or alternation — pass multiple patterns instead).

```bash
# Wait for a state transition, then prove nothing degraded.
native automate assert --timeout-ms 10000 '4 open'
native automate assert --absent 'error event=' 'dispatch_errors=[1-9]'
```

## The generated workflow

`native init --full` writes two jobs for native apps (web-frontend scaffolds get the `test` job only, since the smoke recipe is native-specific):

```yaml
env:
  # build.zig.zon expects the Native SDK framework checkout at this
  # path, relative to the repository root.
  NATIVE_SDK_PATH: "../native-sdk"
```

Both jobs fetch the framework into that path when it is missing, so the app's `build.zig.zon` path dependency resolves on a fresh runner. If your framework checkout lives elsewhere (or you scaffolded from an unusual directory), adjust `NATIVE_SDK_PATH` and `build.zig.zon` together.

The `test` job is tier 1:

```yaml
- uses: vercel-labs/setup-zig@v1
  with:
    version: 0.16.0
- run: zig build test -Dplatform=null
```

The `smoke` job is tier 2. It installs GTK and Xvfb, builds the framework CLI, then builds and drives the app headless:

```yaml
- name: Build and drive the app headless
  run: |
    set -euo pipefail
    cli="$NATIVE_SDK_PATH/zig-out/bin/native"
    zig build -Dplatform=linux -Dweb-engine=system -Dautomation=true
    rm -rf .zig-cache/native-sdk-automation
    xvfb-run -a ./zig-out/bin/my-app &
    pid=$!
    trap 'kill "$pid" >/dev/null 2>&1 || true' EXIT
    "$cli" automate wait
    "$cli" automate assert 'gpu_nonblank=true' 'role=button name="Reset"' 'total: 0'
    "$cli" automate screenshot main-canvas
    test -s .zig-cache/native-sdk-automation/screenshot-main-canvas.png
```

Step by step:

1. `rm -rf .zig-cache/native-sdk-automation` clears stale automation artifacts so assertions never pass against a previous run.
2. `xvfb-run -a` launches the real binary under a virtual display; the `trap` guarantees the app dies with the job.
3. `automate wait` blocks until the runtime publishes `ready=true`.
4. `automate assert` proves the GPU surface presented non-blank pixels and the scaffolded counter UI (the Reset button and the `total: 0` status bar) reached the accessibility snapshot.
5. `automate screenshot` renders the `main-canvas` view to a PNG and the final `test -s` fails the job if it is empty.

As the app grows, extend the assert line with patterns for your own widgets, add `automate widget-click` interactions between assertions, or check screenshots against committed goldens.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)