Skip to content
AutoTestX Docs

GitHub Actions pipeline

Set up a real GitHub repository whose workflow deploys to staging, runs the AutoTestX Smoke suite, and deploys to production only if Smoke passes.

 git push to main
      │
      ▼  GitHub Actions (GitHub's cloud)
 ┌────────────────┐   ┌──────────────────────────────┐   ┌───────────────────┐
 │ deploy-staging │──►│ smoke-test                   │──►│ deploy-production │
 │ (your deploy)  │   │ node autotestx-gate.mjs      │   │ only when Smoke   │
 └────────────────┘   │   POST /api/executions ──────┼─┐ │ passed            │
                      │   GET  …?wait=60 (repeat) ───┼─┤ └───────────────────┘
                      │   exit 0 = pass / 1 = fail   │ │
                      └──────────────────────────────┘ │ HTTPS + API token
                                                       ▼
                                          https://app.autotestx.com
                                          runs Smoke on Staging, emails on failure

You need two files. Download them here, or copy them from the end of this page (The workflow file, The gate script):

They also ship with AutoTestX in examples/github-actions/.

File Goes in your repository as What it is
deploy.yml .github/workflows/deploy.yml The workflow: three jobs
autotestx-gate.mjs .github/autotestx/autotestx-gate.mjs Starts the suite, waits, prints the results, fails the job if the suite fails. One file, no dependencies, needs only Node (every GitHub runner has it).

Why a gate script and not atx run --suite? The AutoTestX CLI ships inside the AutoTestX project and is not published to npm, so an ordinary repository cannot install it. The gate script does the same job in one file you copy in.

Time needed: about 30 minutes.

1. AutoTestX must be reachable from GitHub

Section titled “1. AutoTestX must be reachable from GitHub”

GitHub runs the workflow on its own machines on the internet. They cannot reach http://127.0.0.1:4100 on your computer. Use the hosted AutoTestX at https://app.autotestx.com.

The hosted server must have the new version (deploy details, ?wait=, notifications). Deploy it first, from the project folder:

Terminal window
npm run deploy:full

(deploy:full, because this version adds new npm packages.) Check it:

Terminal window
curl.exe -s -o NUL -w "%{http_code}" https://app.autotestx.com/api/mcp

401 means the new version is live (the MCP endpoint exists and wants a token). 404 means the old version is still running.

Only want to try it on your own computer? Register your PC as a self-hosted runner (repository → Settings → Actions → Runners → New self-hosted runner) and set runs-on: self-hosted. Then ATX_SERVER can be http://127.0.0.1:4100.

2. Prepare AutoTestX (in the browser, on app.autotestx.com)

Section titled “2. Prepare AutoTestX (in the browser, on app.autotestx.com)”
  1. Plan: the organization must be on Team or higher (API tokens are not in the proof of value).
  2. Environment named Staging. In real use it points at your staging site; for this trial its variables do not matter.
  3. Suite Smoke with at least two shared, Ready tests: one passing and one failing on purpose. Open checkout and Apply discount code from the exercises, step 0.7 are ideal: they use a built-in page, so no real website is needed.
  4. API token: Configurations → Project Members → API tokens for CI → name it after the repository, e.g. GitHub Actions - shop-pipeline-demo, role Run tests (Tester), expiry 90 days. Copy the value: it is shown once.
  5. Notifications (Test Suites page): leave Deploy pipeline runs on its default, or add yourself.

3. Create the GitHub repository

Section titled “3. Create the GitHub repository”
  1. On GitHub: New repository → name shop-pipeline-demo → Private → tick Add a README → Create repository.
  2. Secret for the token: repository → Settings → Secrets and variables → Actions → Secrets tab → New repository secret:
    • Name: AUTOTESTX_TOKEN
    • Secret: the atx_pat_… value from step 2.4
  3. Variable for the address: same page → Variables tab → New repository variable:
    • Name: ATX_SERVER
    • Value: https://app.autotestx.com

A secret is stored encrypted, never shown again, and masked as *** in logs. The address is not secret, so it is a plain variable.

4. Add the two files

Section titled “4. Add the two files”

The GitHub website is enough; no Git on your computer is needed.

  1. Repository → Add file → Create new file.
    • Name: .github/autotestx/autotestx-gate.mjs (typing the / creates the folders)
    • Content: paste all of the gate script
    • Commit changes… → commit directly to main.
  2. Again Add file → Create new file:
    • Name: .github/workflows/deploy.yml
    • Content: paste all of the workflow file
    • Commit changes… → main.

The second commit is a push to main, so the workflow starts straight away.

5. Watch the first run (Smoke fails, production is blocked)

Section titled “5. Watch the first run (Smoke fails, production is blocked)”

On GitHub: repository → Actions → the run Deploy.

What you see Meaning
deploy-staging ✅ The (pretend) deploy to staging
smoke-test ❌, red The gate ran the suite and it failed, as intended
deploy-production ⊘ skipped Production was not deployed, because Smoke failed. This is the point of the gate.

Open smoke-test → step AutoTestX Smoke suite on Staging:

AutoTestX: Smoke on Staging - 2 tests · exec_…
Follow it: https://app.autotestx.com/#/suites/exec_…
[FAIL] Apply discount code                      3/4 steps 1.7s
         https://app.autotestx.com/#/runs/run_…
[PASS] Open checkout                            2/2 steps 0.8s


FAILED  suite Smoke - 1/2 passed, 1 failed
Error: AutoTestX Smoke failed: 1/2 passed, 1 failed. https://app.autotestx.com/#/suites/exec_…

The run’s Summary page shows a table, ❌ AutoTestX: Smoke FAILED, with a link to each test’s report.

In AutoTestX: Test Suites → Recent suite runs → the new run is marked deploy. Open it:

Deploy · github-actions · main @ 9f3c2e1a7b4d · #11823456789 · pipeline run Deploy run 1 by your-github-name

These are real values: the commit you just made, your branch, and GitHub’s run number. Click pipeline run: it opens the very GitHub Actions run you were watching.

Email: the people on the Deploy pipeline runs list get “Smoke is failing on Staging after a deploy”, with the same deploy details and a Pipeline: link to the GitHub run.

6. Fix it and re-run (Smoke passes, production deploys)

Section titled “6. Fix it and re-run (Smoke passes, production deploys)”
  1. In AutoTestX, fix the failing test: Apply discount code → New revision → step 4 expected text Code expired → Status Ready → Save.
  2. On GitHub: open the failed run → Re-run jobs → Re-run failed jobs.
  3. Now smoke-test ✅ and deploy-production ✅ runs.
  4. AutoTestX shows a new deploy run that passed. The deploy number ends in .2, the re-run attempt. An email “Smoke is passing again on Staging” goes out.

7. More things to try

Section titled “7. More things to try”
Try Expected
Edit the README on GitHub and commit to main A new run with the new commit. AutoTestX shows that commit.
Actions → Deploy → Run workflow A run without a push (workflow_dispatch)
Cancel a run while smoke-test is running The gate cancels the suite: in AutoTestX the run ends CANCELLED. See Trying a cancel: Smoke is usually too quick to catch.
Change the secret to a wrong value, re-run smoke-test fails with “401 Invalid or revoked API token” (exit code 2)
Set ATX_SUITE: Nope in deploy.yml “404 The suite “Nope” has no shared tests to run.“
Revoke the token in AutoTestX The next run is refused (401). Create a new token and update the secret.

Trying a cancel

Section titled “Trying a cancel”

Cancelling a GitHub run while smoke-test is running should stop the suite in AutoTestX too: the gate sends a cancel, tests that have not started are skipped, tests already running finish, and the suite run ends CANCELLED.

The two Smoke tests take only a few seconds, which is usually too short to click Cancel workflow in time. If the suite has already finished, AutoTestX correctly shows its real result. To try it, add a slow test for a while. Paste this in the browser console on AutoTestX (F12 → Console), with the right project selected:

fetch('/api/tests/import', { method: 'POST', headers: { 'content-type': 'application/json', 'x-project-id': localStorage.getItem('atx.project') },
  body: JSON.stringify({ dslVersion: '1.8.0', name: 'Slow check (40 s)', engine: 'web', module: 'Smoke', status: 'ready',
    defaults: { timeoutMs: 60000, retry: 0, onError: 'abort' },
    steps: [ { id: 'open', type: 'browser.open', params: { url: "data:text/html,<p>working…</p><script>setTimeout(()=>{document.body.innerHTML='<p id=\"done\">done</p>'},40000)</script>" } },
             { id: 'wait', type: 'wait.element', target: { strategies: [{ kind: 'css', value: '#done' }] } } ] }) })
  .then((r) => console.log('Slow check:', r.status === 201 ? 'created' : r.status));

Then start the workflow, open the smoke-test job, and click Cancel workflow while it shows Follow it: …. Expected in the job log:

SIGINT: cancelling exec_… on AutoTestX…
AutoTestX suite run cancelled: tests not yet started will not run; running ones finish, then the suite run ends CANCELLED.

In AutoTestX, the run shows CANCELLED once the running test has finished (up to 40 seconds later). Delete the Slow check test afterwards, so Smoke stays fast.

In the log after cancelling Meaning
The two lines above Working
… it had already finished. The suite ended before the cancel arrived. Its real result stands.
… the cancel request failed (403 State-changing requests must be sent as application/json). An old copy of the gate script. Copy the current one.
No SIGINT: line at all The workflow step does not start with exec. GitHub’s cancel signal stops at the shell. Use run: exec node … as in the workflow file.

8. Using it for a real application

Section titled “8. Using it for a real application”
  • Replace the two echo deploy steps with your real deploy commands.
  • Point the AutoTestX Staging environment at the real staging site, and put real tests in Smoke.
  • Keep Smoke small and fast (10–20 key flows). Run the full regression on an AutoTestX schedule.
  • One token per repository or pipeline, so each can be revoked alone.
  • Optional: protect production with a GitHub environment that requires an approval, so a person confirms even after Smoke passes.

Settings of the gate script

Section titled “Settings of the gate script”
Variable Required Default Meaning
ATX_SERVER yes AutoTestX address
ATX_TOKEN yes API token with the Tester role, from a secret
ATX_SUITE yes The suite to run
ATX_ENVIRONMENT none Environment name or id
ATX_SCOPE ready ready = Ready tests only; all = Drafts too
ATX_TIMEOUT_MIN 30 Cancel the suite and fail after this long
ATX_NOTE One line shown on the suite run
ATX_JUNIT Save the finished suite run as JUnit XML to this file, for a test report step

Read automatically from GitHub Actions: GITHUB_SHA (commit), GITHUB_REF_NAME (branch), GITHUB_RUN_ID and GITHUB_RUN_ATTEMPT (run number), and GITHUB_SERVER_URL + GITHUB_REPOSITORY (the link back). Nothing is sent to GitHub.

Exit codes: 0 passed · 1 failed or error · 2 refused (token, suite, environment, settings) · 130 cancelled.

Troubleshooting

Section titled “Troubleshooting”
In the job log Why Fix
Set ATX_SERVER, ATX_TOKEN… Secret or variable missing or misspelled Check names: secret AUTOTESTX_TOKEN, variable ATX_SERVER
Cannot reach https://…: fetch failed Wrong address, or the server is down Open the address in a browser
401 Invalid or revoked API token Wrong, revoked or expired token New token → update the secret
403 … does not allow running tests A Read-only token Use a Run tests token
404 The suite … has no shared tests Suite name differs, or its tests are private Use the name shown on Test Suites; share the tests
404 Environment "…" not found Environment name differs Use the name shown under Environments
Every test SKIP ATX_SCOPE is ready and no test is Ready Mark the reviewed tests Ready
Deploy line shows no commit The server still runs the old version Deploy the new version (step 1)
429 … 5 suite runs in progress Many runs at once with one token Wait, or keep concurrency: in the workflow

How the pipeline, the API and the emails fit together is explained in Run after deployment and Notifications.

The workflow file

Section titled “The workflow file”

.github/workflows/deploy.yml (download):

# Deploy to staging, gate on the AutoTestX Smoke suite, then deploy to production.
#
# Put this file in your repository as .github/workflows/deploy.yml, and the gate
# script as .github/autotestx/autotestx-gate.mjs. In the repository's
# Settings → Secrets and variables → Actions, add:
#   secret    AUTOTESTX_TOKEN   an AutoTestX API token with the Tester role
#   variable  ATX_SERVER        e.g. https://app.autotestx.com
#
# The two deploy jobs only print a line: replace them with your real deploy.
name: Deploy


on:
  push:
    branches: [main]
  workflow_dispatch: # a "Run workflow" button on the Actions tab


# One deploy at a time per branch; a newer push waits instead of racing the older one.
concurrency:
  group: deploy-${{ github.ref }}
  cancel-in-progress: false


jobs:
  deploy-staging:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Deploy to staging
        run: echo "Deploying ${GITHUB_SHA::7} to staging (replace with your real deploy)"


  smoke-test:
    needs: deploy-staging
    runs-on: ubuntu-latest
    timeout-minutes: 40
    steps:
      - uses: actions/checkout@v4
      - name: AutoTestX Smoke suite on Staging
        # `exec`: node replaces the shell, so when the run is cancelled GitHub's signal reaches
        # the gate and it cancels the suite on AutoTestX. Without it the shell keeps the signal.
        run: exec node .github/autotestx/autotestx-gate.mjs
        env:
          ATX_SERVER: ${{ vars.ATX_SERVER }}
          ATX_TOKEN: ${{ secrets.AUTOTESTX_TOKEN }}
          ATX_SUITE: Smoke
          ATX_ENVIRONMENT: Staging
          ATX_SCOPE: ready
          ATX_TIMEOUT_MIN: '30'
          ATX_NOTE: ${{ github.workflow }} run ${{ github.run_number }} by ${{ github.actor }}


  deploy-production:
    needs: smoke-test # skipped when the Smoke suite fails
    runs-on: ubuntu-latest
    steps:
      - name: Deploy to production
        run: echo "Smoke passed - deploying ${GITHUB_SHA::7} to production (replace with your real deploy)"
Part What it does
on: push: branches: [main] Runs on every push to main. workflow_dispatch adds a Run workflow button.
concurrency One deploy at a time per branch. A newer push waits instead of racing the older one.
deploy-staging Your deploy to staging. Replace the echo with your real deploy.
smoke-test with needs: deploy-staging Starts only after staging is deployed, and runs the gate script
env: of the gate step The settings: the address from a variable, the token from a secret, the suite, environment, scope, time limit and a note
timeout-minutes: 40 GitHub’s own limit for the job. It’s longer than the gate’s 30 minutes, so the gate cancels the suite cleanly first.
deploy-production with needs: smoke-test Starts only when smoke-test succeeded, so a failing Smoke suite blocks production

The gate script

Section titled “The gate script”

.github/autotestx/autotestx-gate.mjs (download). Copy it unchanged.

Show the full script 
#!/usr/bin/env node
/**
 * AutoTestX deploy gate - run a suite after a deploy and fail the job if it does not pass.
 *
 * One file, no dependencies, Node 18 or newer (every GitHub-hosted runner has it).
 * Copy it into your repository, e.g. .github/autotestx/autotestx-gate.mjs, and call it
 * from a workflow step (in Jenkins, from an sh step: see examples/jenkins/Jenkinsfile):
 *
 *   - run: node .github/autotestx/autotestx-gate.mjs
 *     env:
 *       ATX_SERVER: https://app.autotestx.com
 *       ATX_TOKEN: ${{ secrets.AUTOTESTX_TOKEN }}
 *       ATX_SUITE: Smoke
 *       ATX_ENVIRONMENT: Staging
 *
 * Settings (environment variables):
 *   ATX_SERVER        AutoTestX address                         (required)
 *   ATX_TOKEN         API token with the Tester role            (required, from a secret)
 *   ATX_SUITE         Suite to run                              (required)
 *   ATX_ENVIRONMENT   Environment name or id                    (optional)
 *   ATX_SCOPE         "ready" (default) or "all"
 *   ATX_TIMEOUT_MIN   Give up and cancel after this long        (default 30)
 *   ATX_NOTE          One line shown on the suite run           (optional)
 *   ATX_JUNIT         Save the suite run as JUnit XML here      (optional, e.g. smoke.xml)
 *
 * The commit, branch, run number and a link back to this run are read from the
 * variables GitHub Actions sets on every job (GITHUB_SHA, GITHUB_REF_NAME, ...), or
 * Jenkins on every build (GIT_COMMIT, GIT_BRANCH, BUILD_NUMBER, BUILD_URL).
 * Nothing is sent to GitHub or Jenkins; they are only reported to AutoTestX.
 *
 * Exit codes: 0 passed · 1 failed or error · 2 refused (token, suite, environment) · 130 cancelled
 */
import { appendFileSync, writeFileSync } from 'node:fs';


const env = process.env;
const server = (env.ATX_SERVER ?? '').replace(/\/+$/, '');
const token = env.ATX_TOKEN ?? '';
const suite = env.ATX_SUITE ?? '';
const scope = env.ATX_SCOPE || 'ready';
const timeoutMs = Number(env.ATX_TIMEOUT_MIN || 30) * 60_000;
// GitHub turns "::error::" lines into annotations; anywhere else they would just be noise.
const ERROR = env.GITHUB_ACTIONS === 'true' ? '::error::' : 'Error: ';


const missing = [!server && 'ATX_SERVER', !token && 'ATX_TOKEN', !suite && 'ATX_SUITE'].filter(Boolean);
if (missing.length) {
  console.error(`${ERROR}Set ${missing.join(', ')} for the AutoTestX gate.`);
  process.exit(2);
}


/** What GitHub Actions or Jenkins says about this run. Empty values are left out. */
function deployment() {
  if (env.GITHUB_ACTIONS !== 'true' && env.JENKINS_URL) {
    const d = { source: 'jenkins' };
    if (env.GIT_COMMIT) d.commit = env.GIT_COMMIT;
    if (env.GIT_BRANCH || env.BRANCH_NAME) d.branch = env.GIT_BRANCH || env.BRANCH_NAME;
    if (env.BUILD_NUMBER) d.deployId = env.BUILD_NUMBER;
    if (env.BUILD_URL) d.url = env.BUILD_URL;
    if (env.ATX_NOTE) d.note = env.ATX_NOTE.slice(0, 300);
    return d;
  }
  const d = { source: env.GITHUB_ACTIONS === 'true' ? 'github-actions' : (env.ATX_SOURCE || 'pipeline') };
  if (env.GITHUB_SHA) d.commit = env.GITHUB_SHA;
  if (env.GITHUB_REF_NAME) d.branch = env.GITHUB_REF_NAME;
  if (env.GITHUB_RUN_ID) d.deployId = `${env.GITHUB_RUN_ID}${env.GITHUB_RUN_ATTEMPT && env.GITHUB_RUN_ATTEMPT !== '1' ? `.${env.GITHUB_RUN_ATTEMPT}` : ''}`;
  if (env.GITHUB_SERVER_URL && env.GITHUB_REPOSITORY && env.GITHUB_RUN_ID) {
    d.url = `${env.GITHUB_SERVER_URL}/${env.GITHUB_REPOSITORY}/actions/runs/${env.GITHUB_RUN_ID}`;
  }
  if (env.ATX_NOTE) d.note = env.ATX_NOTE.slice(0, 300);
  return d;
}


class Refused extends Error {}


async function call(method, path, body, timeoutMs = 150_000) {
  let res;
  try {
    res = await fetch(`${server}/api${path}`, {
      method,
      // Every POST is sent as JSON, even one with nothing to say (a cancel): the server
      // refuses a state-changing request that is not application/json.
      headers: { authorization: `Bearer ${token}`, ...(method === 'GET' ? {} : { 'content-type': 'application/json' }) },
      ...(method === 'GET' ? {} : { body: JSON.stringify(body ?? {}) }),
      signal: AbortSignal.timeout(timeoutMs),
    });
  } catch (e) {
    throw new Error(`Cannot reach ${server}: ${e instanceof Error ? e.message : e}`);
  }
  const text = await res.text();
  let data = null;
  try {
    data = text ? JSON.parse(text) : null;
  } catch {
    /* a proxy's error page */
  }
  if (!res.ok) {
    const msg = `${res.status} ${data?.error ?? res.statusText}${data?.code ? ` (${data.code})` : ''}`;
    // 4xx: the request is wrong (token, suite, environment) - a person has to fix it.
    if (res.status >= 400 && res.status < 500) throw new Refused(msg);
    throw new Error(msg);
  }
  return data;
}


const MARK = { passed: 'PASS', failed: 'FAIL', error: 'ERR ', skipped: 'SKIP', cancelled: 'STOP', queued: 'WAIT', running: 'RUN ' };
const seconds = (ms) => `${((ms ?? 0) / 1000).toFixed(1)}s`;


let executionId = null;
let stopping = false;


// A cancelled job (or Ctrl+C) stops the suite on the server instead of leaving it running.
// GitHub sends SIGINT, then SIGTERM about 7.5 s later, then kills the process: the
// cancel request gets 5 s, so it is sent before the process is killed.
// The process must receive the signal itself: start it with `exec node …` in the
// workflow, because bash does not pass a signal on to the command it is running.
async function cancelAndExit(signal) {
  if (stopping) return;
  stopping = true;
  if (!executionId) {
    console.log(`${signal}: stopping before a suite was started.`);
    process.exit(130);
  }
  console.log(`${signal}: cancelling ${executionId} on AutoTestX…`);
  const result = await call('POST', `/executions/${executionId}/cancel`, undefined, 5_000).then(
    () => 'cancelled: tests not yet started will not run; running ones finish, then the suite run ends CANCELLED.',
    (e) => (e instanceof Refused && e.message.startsWith('409') ? 'it had already finished.' : `the cancel request failed (${e.message}).`),
  );
  console.log(`AutoTestX suite run ${result}`);
  process.exit(130);
}
process.on('SIGINT', () => void cancelAndExit('SIGINT'));
process.on('SIGTERM', () => void cancelAndExit('SIGTERM'));


function summary(e) {
  const file = env.GITHUB_STEP_SUMMARY;
  if (!file) return;
  const icon = e.status === 'passed' ? '✅' : e.status === 'cancelled' ? '⏹️' : '❌';
  const rows = e.children
    .map((c) => `| ${c.status === 'passed' ? '✅' : c.status === 'skipped' ? '⏭️' : c.status === 'cancelled' ? '⏹️' : '❌'} | ${c.runId ? `[${c.testName}](${server}/#/runs/${c.runId})` : c.testName} | ${c.status}${c.reason ? ` - ${c.reason}` : ''} | ${c.total !== undefined ? `${c.passed ?? 0}/${c.total}` : ''} |`)
    .join('\n');
  appendFileSync(
    file,
    `### ${icon} AutoTestX: ${e.suite} ${e.status.toUpperCase()}\n\n` +
      `${e.counts.passed}/${e.counts.total} passed · [open the suite run](${e.url ?? `${server}/#/suites/${e.executionId}`})\n\n` +
      `| | Test | Result | Steps |\n|---|---|---|---|\n${rows}\n`,
  );
}


/** ATX_JUNIT: the finished suite run as JUnit XML, for the CI system's test report step. */
async function saveJunit() {
  if (!env.ATX_JUNIT) return;
  try {
    const res = await fetch(`${server}/api/executions/${executionId}/report.junit.xml`, {
      headers: { authorization: `Bearer ${token}` },
      signal: AbortSignal.timeout(60_000),
    });
    if (!res.ok) throw new Error(`${res.status} ${res.statusText}`);
    writeFileSync(env.ATX_JUNIT, await res.text());
    console.log(`JUnit XML saved to ${env.ATX_JUNIT}`);
  } catch (err) {
    // The suite's result decides the build, not the report file.
    console.log(`Could not save the JUnit XML (${err instanceof Error ? err.message : err}).`);
  }
}


async function main() {
  const started = await call('POST', '/executions', {
    suite,
    scope,
    ...(env.ATX_ENVIRONMENT ? { environment: env.ATX_ENVIRONMENT } : {}),
    deployment: deployment(),
  });
  executionId = started.executionId;
  console.log(`AutoTestX: ${suite}${env.ATX_ENVIRONMENT ? ` on ${env.ATX_ENVIRONMENT}` : ''} - ${started.tests} tests · ${executionId}`);
  console.log(`Follow it: ${started.url}`);


  const deadline = Date.now() + timeoutMs;
  const printed = new Set();
  let e;
  for (;;) {
    e = await call('GET', `/executions/${executionId}?wait=60`).catch((err) => {
      if (err instanceof Refused) throw err;
      console.log(`(retrying after: ${err.message})`);
      return null;
    });
    if (e) {
      for (const c of e.children) {
        if (printed.has(c.testId) || c.status === 'queued' || c.status === 'running') continue;
        printed.add(c.testId);
        console.log(`[${MARK[c.status] ?? c.status}] ${c.testName.padEnd(40)} ${c.total !== undefined ? `${c.passed ?? 0}/${c.total} steps` : ''} ${c.durationMs !== undefined ? seconds(c.durationMs) : ''}`);
        if (c.reason) console.log(`         ${c.reason}`);
        if ((c.status === 'failed' || c.status === 'error') && c.runId) console.log(`         ${server}/#/runs/${c.runId}`);
      }
      if (e.status !== 'running') break;
    }
    if (Date.now() > deadline) {
      console.error(`${ERROR}AutoTestX suite did not finish within ${timeoutMs / 60_000} minutes - cancelling it.`);
      await call('POST', `/executions/${executionId}/cancel`).catch(() => {});
      return 1;
    }
  }


  summary(e);
  await saveJunit();
  const c = e.counts;
  const tally = `${c.passed}/${c.total} passed${c.failed ? `, ${c.failed} failed` : ''}${c.error ? `, ${c.error} error` : ''}${c.skipped ? `, ${c.skipped} skipped` : ''}`;
  if (e.status === 'passed') {
    console.log(`\nPASSED  suite ${e.suite} - ${tally} in ${seconds(e.durationMs)}`);
    return 0;
  }
  console.log(`\n${e.status.toUpperCase()}  suite ${e.suite} - ${tally}`);
  console.error(`${ERROR}AutoTestX ${e.suite} ${e.status}: ${tally}. ${e.url ?? ''}`);
  return e.status === 'cancelled' ? 130 : 1;
}


main()
  .then((code) => process.exit(code))
  .catch((err) => {
    console.error(`${ERROR}AutoTestX: ${err.message}`);
    process.exit(err instanceof Refused ? 2 : 1);
  });

What it does, in order:

  1. Checks that ATX_SERVER, ATX_TOKEN and ATX_SUITE are set (otherwise exit 2).
  2. Builds the deploy details from GitHub’s variables: GITHUB_SHA becomes the commit, GITHUB_REF_NAME the branch, GITHUB_RUN_ID (plus the attempt) the run number, and server, repository and run id make the link.
  3. Sends POST /api/executions with the suite, environment, scope and deploy details.
  4. Repeats GET /api/executions/:id?wait=60 until the suite finishes, printing each test once.
  5. Writes a results table to the GitHub run’s Summary page (GITHUB_STEP_SUMMARY).
  6. Exits 0 when the suite passed, 1 when it failed, 2 when AutoTestX refused the request.
  7. If the job is cancelled, cancels the suite on AutoTestX first (exit 130). After ATX_TIMEOUT_MIN minutes, cancels it and fails.