Command line (`atx`)

atx runs tests from the command line with the same engine as the server. It is built for CI pipelines: one command, a machine-readable result, and a non-zero exit code on failure. It works on test files, so it needs no server.

From the project folder:

npm run atx -- <command> [options]

npm run atx -- passes everything after -- to atx. If your shell swallows the -- or the --option flags (some PowerShell setups do), call the CLI directly:

node --import tsx packages/cli/src/index.ts run test.json --env env.json

Commands

Command	Does
atx validate <test.json>	Checks the file against the schema, the step registry and the expression parser
atx migrate <test.json> [--write]	Migrates the file to the current DSL version. Prints the result, or overwrites the file with --write.
atx run <test.json> [options]	Runs the test in Chromium
atx help	Usage

Exit codes

Code	Meaning
0	Passed (or valid)
1	Failed (or invalid, for validate)
2	Invalid input: unreadable file, invalid test, missing data columns, bad usage

atx run options

Option	Meaning
--env <file.json>	Environment variables and secrets (see below)
--artifacts <dir>	Write screenshots and other artifacts here. Data-driven rows go in <dir>/r1, r2, …
--screenshots	Capture a screenshot after every step (default: failures and Screenshot steps only)
--headed	Show the browser. Data-driven rows then run one at a time.
--files <dir>	Folder that relative Upload file paths resolve from. Default: the test file’s folder.
--json <file>	Write the machine-readable result
--data <file>	Rows for a data-driven run: a CSV with a header row, or a JSON list of objects
--concurrency <n>	Rows at once, when the test declares its rows independent

Environment file

{
  "name": "Fixture UAT",
  "variables": {
    "baseUrl": "https://uat.example.com"
  },
  "secrets": {
    "APP_USER": "tester",
    "APP_PASSWORD": "correct horse"
  }
}

variables are the environment variables ({{baseUrl}}). secrets are {{secret.NAME}}, masked in the output. Do not commit real secrets. Generate this file in CI from your secret store.

Examples

Validate:

npm run atx -- validate examples/create-purchase-order.json

Run with an environment and keep the result:

npm run atx -- run examples/create-purchase-order.json --env examples/fixture.env.json --json result.json

examples/fixture.env.json points baseUrl at the fixture app with an absolute file:/// path. Edit it to your project folder first.

Output (shortened):

[PASS] Enter username                            400 ms
[PASS] Enter password                            345 ms
[PASS] Sign in                                   387 ms
[PASS] Purchase order list is shown               19 ms
…
[PASS] Add each order line                      2010 ms
[PASS] Save the purchase order                   342 ms
[PASS] Extract the generated PO number            17 ms
[PASS] Verify a PO number was generated            0 ms
[PASS] Verify the status is Approved              16 ms
[PASS] The new order appears in the list          21 ms

PASSED  Create Purchase Order
18 / 18 steps passed in 4427 ms

Other markers you may see:

Marker	Meaning
[FAIL]	The step failed. The error is printed on the next line.
[WARN] … (degraded: css=#btn_new_412)	Passed using a fallback locator
[RTRY] attempt 1: …	A retry, with the reason
[SKIP]	Not run
n soft assertion failure(s)	Soft assertions failed; the run fails

Data-driven, with rows from a CSV, three at a time:

npm run atx -- run examples/data-driven/02-dataset-order-totals-parallel.json --env examples/api-demo/environment.json --data examples/data-driven/data/order-lines.csv --concurrency 3

Each line is prefixed with its row (r 3). At the end, every failed row is listed with its inputs (sensitive columns as ***) and the step that failed.

Data-driven tests in the CLI

Test’s data source	What the CLI uses
Inline table	The table, unless --data is given
Data set, HTTP request, data pool	These live on the server. Pass the rows with --data. Without it the run stops with an explanation.

If a step uses {{data.x}} and the rows have no column x, the run stops before starting (exit code 2).

Things the CLI does not do

• It does not talk to a server. Components called by a test (component.call) are resolved by the server, so a test that calls components must run on the server.
• It has no project, environment or secret store. Everything comes from files.

In a CI pipeline

# GitHub Actions example
- run: npm ci
- run: npx playwright install --with-deps chromium
- run: npm run atx -- run tests/create-po.json --env env.json --json result.json --artifacts artifacts
- uses: actions/upload-artifact@v4
  if: always()
  with: { name: autotestx, path: "result.json\nartifacts" }

Write env.json from your CI secret store in a step before the run.