运行

Configuration

A tour of danwo.config.ts, grouped by what each option is for.

本页还没有中文版,下面是英文原文。

danwo.config.ts default-exports one object. Every key is optional; defineConfig only types it.

danwo.config.ts
import { defineConfig, stripeEnvironment } from "@danwo/sdk"

export default defineConfig({
  testDir: "tests",
  workers: 2,
  retries: 1,
  model: "anthropic/claude-sonnet-5",
  readiness: { mode: "page+backend", settle: true },
  video: "retain-on-failure",
  environment: () => stripeEnvironment(),
})

This page groups the options. Reference: Config lists each one with its type.

The config loads like a test file: Node removes its types and compiles nothing, so an enum or a namespace won't load. TypeScript in tests lists what works, imports included.

Types aren't checked either, so a misspelt key would do nothing. danwo warns about each key it doesn't know, and suggests the one you likely meant:

danwo: warning: unknown config key `retires` (did you mean `retries`?); danwo ignores it

Finding and running tests

OptionDefaultWhat it does
testDirthe config's folderwhere test files are found
testMatchevery *.danwo.tsglobs relative to testDir
workers1worker processes, each with its own environment
splitFilesfalsehand workers single tests, not whole files, where that is safe
retries0extra attempts for a failing test
timeout60_000one attempt's deadline, in milliseconds
baseUrlnonethe app's URL when no environment gives one

Split slow files across workers

A worker runs one test file at a time, so one long file can keep one worker busy while the others sit idle. splitFiles: true (or danwo test --split-files) hands out single tests instead.

danwo only splits a file whose tests share nothing: no beforeAll or afterAll, and every test on isolation: "test". Other files still run whole. Each test loads its file again, so tests can't pass values to each other through variables at the top of the file.

The order tests run in

danwo promises an order, so a suite whose tests build on each other can rely on it:

  • Files start in path order, compared character by character: 10- sorts before 2-, so pad numbers, and capitals sort before lower case.
  • A file's tests run in the order they're declared, one after another in one worker. A retry runs straight after the attempt it repeats.
  • With workers: 1, each file finishes before the next one starts.

With more workers, files still start in that order but run side by side. Split files hand their tests to different workers, so keep splitFiles off for tests that rely on each other.

Give each worker its own port

Each worker starts its own environment, so two workers can't start the app on one fixed port. freePort() picks a port nothing listens on. It comes from a range the system doesn't hand out to other programs, and no other worker or danwo run gets the same one, so the port is still free when the app starts:

danwo.config.ts · a port per worker
import { appEnvironment, defineConfig, freePort } from "@danwo/sdk"

export default defineConfig({
  workers: 2,
  async environment() {
    const port = await freePort()

    return appEnvironment({ app: { command: "npm start", url: `http://127.0.0.1:${port}` } })
  },
})

The app gets the port as PORT.

Start an app without the proxy

appEnvironment starts the app behind danwo's proxy, so its calls to providers reach the emulators. Some apps must not be routed at all, such as a tool that runs danwo itself. Pass proxy: false to start the app as it would run on its own:

danwo.config.ts · an app without the proxy
environment: () =>
  appEnvironment({
    app: { command: "npm start", url: "http://127.0.0.1:4310" },
    proxy: false,
  }),

The app then gets none of the proxy's settings. With no proxy there are no emulators, so adding services as well stops the run with a config error.

What tests run against

OptionDefaultWhat it does
environmentnonestarts the app, emulators and proxy, once per worker
isolation"test"reset to the seeds before every test, once per "file", or "none"
sessionsnonesign-ins set up once per worker and reused (Saved sessions)
secretsnonevalues the model never sees (Secrets)
filesnonefiles tests may upload, by name
browserChromiumthe engines tests run in, and the browser's locale, time zone and size

Choose the browsers

browser.name picks the engine: Chromium, Firefox or WebKit. A list runs every test once in each, and labels each result with its engine:

danwo.config.ts · the browsers
browser: {
  name: ["chromium", "webkit"],
  contextOptions: { locale: "en-GB" },
},
KeyDefaultWhat it does
name"chromium"one engine, or a list of them
contextOptionsnonelocale, time zone, colour scheme, size, HTTP sign-in, headers

danwo test --browser firefox replaces name for one run. Install each engine with npx @danwo/cli install-browser --browser firefox,webkit. Web shows what works in each engine.

Waiting

OptionDefaultWhat it does
readiness"page+backend" with a proxyafter each action, wait for the page and the app's calls
appErrors{ console: false }console: true also fails a step on console errors

The agent

OptionDefaultWhat it does
modelnonean AI SDK model, or a Vercel AI Gateway id
judgemodelthe model agent.assert judges with
judgeScreenshotfalseshow the judge a screenshot too
vision"auto"when the agent may see the page as an image
budget25 turns a steplimits on turns, calls, dollars and time

Replays

OptionDefaultWhat it does
cachenone: no cache{ dir }, where the replay cache lives
relocation"similar-v1"find an element that moved, or "exact" never does
effectstruea replayed step must change what its recording changed

Records and reports

OptionDefaultWhat it does
record"always", 50 runsa flight record mode, or { mode, keepRuns } to keep more runs
video"on"keep video with each record, "retain-on-failure", or "off"; { mode, cursor } too
reportersnonee.g. github() from @danwo/github (CI)
linton, never failsintegration lint levels and failOn
impactnonerules for changed tests

The fault matrix

matrix holds the matrix's own settings:

OptionDefaultWhat it does
budget40most faulted trials a matrix runs
workersworkers, else one per core (up to 8)matrix worker processes
racesall ondouble submits, early and reordered webhooks, slow retries
outboundFaultseach provider's ownthe faults planned on each provider call
webhookFaultsthe defaultsthe faults planned on each webhook
dataFaultsthe defaultsfaults on the app's database and cache calls
rerunInvalid0reruns of a trial whose fault never fired
warmUpfalseruns the test once before the clean run, for a warm app
checkPace3× the clean run, at least 1 s; on a device 6 s, once the fault firedhow long a faulted run's checks wait

See Planning and budget for how the budget is spent.

Next

本页内容

在每个 Pull Request 上运行

每个 Pull Request 一个 GitHub Check 和一条评论,并附上完整的运行记录。

配置 CI