Tests in CI

There are two lanes. The managed lane is the one to set up for a team: your repo keeps zero secrets, and results land everywhere your team looks. The manual lane is plain exit codes for any CI system.

#The managed lane: tests on every pull request

If your repo uses the GitHub App for PR previews, tests are one input away. The workflow step that runs the preview also runs your Maestro flows:

- uses: sootbean/soot-preview-action@v1
  with:
    start: npm run dev
    port: 8081
    maestro: auto

maestro: auto discovers every flow in your repo’s .maestro/ directory. You can also pass explicit paths, comma-separated, and control which events run tests with maestro-on (default pull_request; add push:main style entries for branch builds):

maestro: .maestro/login.yaml,.maestro/checkout.yaml
maestro-on: pull_request,push:main

Each flow runs against the same app the preview booted, records itself, and uploads its own recording. On every PR you get:

• +
  A comment with a Tests table: each flow, its result, and a recording link.
• +
  A “Contrast Tests” check on the commit whose conclusion is the real aggregate, so a failing flow blocks merges under branch protection.
• +
  Dashboard rows: every flow indexed in your dashboard with branch, commit, PR, and a per-step trace.

The repo adds no API keys and no secrets; the workflow authenticates with its own GitHub token. A failing flow marks the check red and the run failed without aborting the preview itself. This lane requires the org’s plan to include previews; see Plans.

Setup from zero: install the GitHub App and accept the bootstrap PR, per GitHub PR Previews.

#The manual lane: any CI, exit codes

Test runs are free and account-less: the runner exits nonzero when any flow or spec fails, so any CI gates on the exit code.

name: rnx tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: oven-sh/setup-bun@v2
      - run: bun install
      - run: curl -fsSL https://rnxsim.com/install.sh | RNX_NO_MODIFY_PATH=1 sh
      - run: bunx playwright install chromium

      - run: bun dev &
      - run: |
          for i in $(seq 1 60); do
            curl -sf http://localhost:8081 > /dev/null && break
            sleep 2
          done

      - run: rnx open 8081 --driver playwright --headless --no-describe
      - run: rnx maestro test .maestro/

The same shape works on any CI system: your app toolchain, Playwright’s Chromium, your dev server, and the standalone rnx executable are the whole dependency list. The installer verifies the executable and does not register the background daemon. rnx open installs the runtime when needed, starts the lightweight bridge for the job, and the operating system removes it with the rest of the job environment.

To ALSO get recordings and dashboard rows from the manual lane, add a RNX_API_KEY secret (created in the rnx account UI, on the account that should own the runs) and pass --preview:

- run: rnx maestro test .maestro/ --preview
  env:
    RNX_API_KEY: ${{ secrets.RNX_API_KEY }}

Runs registered from CI carry their git provenance automatically (branch, commit, PR number, workflow run) so the dashboard can group and link them.

#Next

• +
  Read the results
• +
  GitHub PR Previews setup