Prompt guide: writing tests with AI

This page shows testers what to tell an AI so that the test it writes for AutoTestX is right the first time: real step types, real elements, exact expected results.

Worked examples: AI prompt examples has seven prompts, each with the test it produced. Every one passes validation and passes when run.

There are two ways to use AI with AutoTestX. Pick one first, because the prompts differ.

	Option A: Write with AI (in the editor)	Option B: an AI chat (Claude or similar)
You write	Plain-language steps	A structured prompt that asks for a DSL JSON file
You get	Steps added to the open test	A complete test file (`.json`)
Knows your project	Yes: environment, secret names, elements your saved tests use, your team’s accepted phrasings	Only what you paste into the prompt
Loops, If / Else, lookups	No, only flat steps	Yes
Data-driven, cleanup, API setup	No	Yes
Best for	Most UI tests, adding steps to an existing test	Complex flows, API tests, data-driven tests, converting many manual cases
How it gets in	Add n steps, then Save	`atx validate`, then import (see Getting the file in)

Rule of thumb: start with Option A. Switch to Option B when the test needs a loop, a condition, a data set, API calls or cleanup.

1. What the AI needs to know

Whichever option you use, a good prompt answers these questions. If a new colleague would have to ask you something before automating the test, put the answer in the prompt.

#	Information	Example	Without it the AI will…
1	Application and screen	“Maximo, Purchase Orders screen”	guess where to start
2	Environment	`{{baseUrl}}`, or `{{baseUrl}}/orders`	hard-code a URL
3	Credentials, by secret name only	`{{secret.APP_USER}}`, `{{secret.APP_PASSWORD}}`	invent names, or worse, use real values
4	Preconditions	“Vendor Globex exists and is Active”	fail on missing data
5	Steps in order, one action each	“Click New Purchase Order”	merge or skip actions
6	Exact element names as shown on screen	button “Save”, field label “Quantity”, test id `save-po`	guess targets that do not resolve
7	Test data	vendor `Globex`, item `1234`, qty `2`	make up values
8	Expected results, exact text	status shows exactly “Approved”; URL contains `/orders/`	write weak checks, or none
9	Values to remember	“Remember the PO number as `poNumber`”	lose the generated value
10	What must not stop the test	“Status check is soft”	abort on a minor check
11	Cleanup (Option B)	“Delete the created order via API, accept 204 or 404”	leave orphan data

Never paste real passwords, tokens or customer data into any prompt. Refer to secrets by name; the values stay in the environment.

2. Option A: prompts for Write with AI

Write with AI already knows the step catalogue, your environment’s variable and secret names, and elements your project’s saved tests use. Your job is to describe the actions and checks clearly.

How to phrase it

One action per line. “Click Save” and “Check the status” are two lines.
Use the words on the screen, in quotes. Click "New Purchase Order", not “create a PO”.
Say what kind of element it is when it helps: button, link, field, dropdown, checkbox, tab.
Start every check with Check that and give the exact expected text.
Name secrets: “sign in with APP_USER and APP_PASSWORD”.
Say “remember … as name” for a value later steps need, then use {{name}}.
Say “soft” for a check that should be reported but not stop the test.
Leave out loops, If / Else and lookups. Add those in the editor afterwards.

Weak, better, best

Weak. Too vague. The AI has to guess the screens, the data and what “works” means.

Test creating a purchase order.

Better. The steps are there, but the elements and checks are still loose.

Log in, create a purchase order for Globex with item 1234 qty 2, save it and check it worked.

Best. One action per line, screen wording, named secrets, exact checks.

Open the application.
Type {{secret.APP_USER}} into the "Username" field.
Type {{secret.APP_PASSWORD}} into the "Password" field.
Click the "Sign In" button.
Check that the heading "Purchase Orders" is visible.
Click the "New Purchase Order" button.
Select "Globex" in the "Vendor" dropdown.
Type 1234 into the "Item" field.
Type 2 into the "Quantity" field.
Click "Add Item".
Click "Save".
Remember the text of the PO number (test id po-number) as poNumber.
Check that poNumber starts with "PO-".
Check that the status (test id po-status) is exactly "Approved" - soft check.
Take a screenshot named "po-saved".

Fill-in template for Write with AI

Open {{baseUrl}}<path>.
<sign-in lines, using {{secret.NAME}} for credentials>
<for each action:>
  <Click | Type <value> into | Select <option> in | Tick | Hover over | Press <key> in> the "<screen wording>" <button/field/link/...>.
<for each check:>
  Check that <element> <is visible | is not visible | shows exactly "<text>" | contains "<text>" | has value "<value>">.
  Check that the URL contains "<part>".
<for values to reuse:>
  Remember the <text | value> of <element> as <variableName>.

After generating: read each step’s from “…” line, untick anything wrong, read the Notes (assumptions and skipped text), choose Add n steps, then use Pick to confirm any target your project has not used before. Run once, then Save.

3. Option B: prompts for an AI chat that writes the test file

An AI chat does not know AutoTestX. The prompt has to teach it the test file format and give it your application’s details. Copy the master prompt below, fill in the <…> parts, and paste real material (screen HTML, an existing test, an API sample) wherever possible.

Master prompt

You are an expert test automation engineer writing tests for AutoTestX.
AutoTestX tests are JSON documents in its own DSL, NOT Playwright/Selenium code.
Output one valid AutoTestX test file.

## DSL rules (follow exactly)
- Top level: dslVersion "1.8.0", id (tc_<snake_case>), name, description,
  engine "web", module, status "draft", tags[],
  defaults { timeoutMs, retry, onError: "abort" }, variables {}, steps [], cleanup [].
- Each step: id (unique kebab-case), type, label (plain-English), optional target,
  params, optional soft: true, optional onError: "continue".
- Allowed step types ONLY:
  browser.open {url} | browser.back | browser.forward | browser.refresh
  click | doubleClick | hover | clear
  input {value, clearFirst?, sensitive?} | select {value, by: value|label|index}
  check {checked} | upload {files} | keyboard {key}
  assert.exists {negate?} | assert.visible {negate?}
  assert.text {expected, match: equals|contains|regex, negate?}
  assert.value {expected} | assert.attribute {name, expected}
  assert.url {expected, match} | assert.expression {expression}
  wait.element {state: visible|hidden|attached|detached} | wait.networkIdle
  screenshot {name, fullPage?}
  control.if {condition} (children, orElse) | control.forEach {items, as} (children)
  control.repeat {times, as?} | control.while {condition, maxIterations}
  extract {saveAs, from: text|value|attribute} | setVariable {name, value}
  http.request {url, method, headers, body, expectStatus, saveAs, sensitive?}
  http.poll {url, until, intervalMs, maxAttempts}
- Targets: "target": { "strategies": [ ... ] } listing 1-3 strategies, most durable first:
  testId > role (+ name) > label > placeholder > text > css. Never xpath.
  Add "confidence" 0-1 to each when giving several.
  Rows/containers: "scope": { "within": { "role": "row", "key": "{{poNumber}}" } }.
  Each target must match EXACTLY ONE element (inside a row: "td.status" or
  "td:first-child", not "td").
- Templates: {{baseUrl}}, {{variable}}, {{secret.NAME}}, {{data.column}}, {{item.field}}.
  Templates work in step params and in scope.within.key, but NOT in a strategy's
  value or name: those are literal. To find an element by a variable, put the
  variable in scope.within.key or check it with "expected".
  Any input that types a secret gets "sensitive": true.
- "expected" can never be empty. To check a field is empty: extract it, then
  assert.expression LENGTH(x) == 0.
- Before a negative check ("is gone", "is not visible"), first prove the target
  resolves with a positive check, so the negative check cannot pass by accident.
- Expressions (assert.expression, setVariable, conditions) use functions:
  CONCAT, CONTAINS, STARTS_WITH, ENDS_WITH, LENGTH, UPPER, LOWER, TRIM, REPLACE,
  SUBSTRING, SPLIT, JOIN, REGEX_MATCH, REGEX_EXTRACT, NUMBER, ROUND, ABS, MIN, MAX,
  COUNT, FIRST, LAST, NOW, DATE_ADD, FORMAT_DATE, PAD_START. Operators: == != < > AND OR NOT.
  Typing is strict: compare numbers with NUMBER(...) when the value is text.
- Never use fixed "wait" for timing; use wait.element or an assertion.
- Non-critical checks get "soft": true. Every cleanup step gets "onError": "continue";
  API deletes use expectStatus "204,404".
- Records the test creates get unique names, e.g. setVariable CONCAT('Auto PO ', NOW()).
- Do NOT invent element names, test ids, endpoints or secret names. If something
  is unknown, use a value starting with "TODO-" and list it after the JSON.

## Example of the house style
<paste one existing test from the DSL tab (Copy), e.g. examples/create-purchase-order.json>

## The test to write
Application / screen: <e.g. Maximo - Purchase Orders>
Environment variables available: baseUrl<, apiUrl, ...>
Secret names available: <APP_USER, APP_PASSWORD, API_TOKEN ...>
Preconditions: <...>
Requirement / acceptance criteria:
<paste user story, AC or manual test case>

Screen details (elements to use):
<paste HTML of the relevant part of the page, OR a list like:
 - button "New Purchase Order", data-testid="new-po"
 - field label "Vendor" (dropdown)
 - text data-testid="po-status">

Test data: <values; or "data-driven: columns vendor, item, qty">
Expected results: <exact texts, URLs, statuses>
Values to remember for later steps: <...>
Cleanup needed: <what to delete and how>

## Output
1. If required information is missing, ASK ME first. Do not guess.
2. A short table: step id | action | target | expected result.
3. The complete test JSON in one ```json block.
4. A list of assumptions and every TODO- value I must fill in.

Variant: UI test with a loop or condition

Add to The test to write:

Order lines are data: put them in "variables": { "lines": [ {item, qty}, ... ] }
and add them with control.forEach (items "lines", as "line").
If the "Confirm" dialog appears (role dialog, name "Confirm"), click "OK";
use control.if with a condition, not a fixed wait.

Worked example: example 3.

Variant: API test

This is an API test with http.request steps (it may also open the UI to verify).
Base API URL: {{apiUrl}}. Auth: POST {{apiUrl}}/auth/token with
{{secret.API_USER}} / {{secret.API_PASSWORD}}, token at auth.body.token,
mark that step "sensitive": true.
Endpoint under test: <METHOD /path>
Request sample: <paste JSON>
Response samples: <201 {...}, 400 {...}, 401 {...}>
Business rules: <e.g. quantity 1-99, vendor must exist>
Check: status via expectStatus; body fields via assert.expression on <saveAs>.body...
For asynchronous jobs use http.poll with an "until" expression on response.
Cleanup: DELETE what was created, expectStatus "204,404", onError "continue".

Worked example: example 7. See also API testing and the worked files in examples/api-demo.

Variant: data-driven test

Run once per row. Add "data": {
  "source": { "kind": "inline", "columns": [<cols>], "rows": [<rows>] },
  "independent": <true if rows do not affect each other>,
  "sensitiveColumns": [<e.g. password>] }
Use {{data.<column>}} in steps. Include rows for: valid case, each invalid
case with its exact error message, and boundary values <list>.

To use a project data set instead, ask for "source": { "kind": "dataSet", "id": "<tds_… id>" }. Worked example: example 5. See also Data-driven testing.

Variant: convert manual test cases

Convert each manual test case below into a separate AutoTestX test file.
Keep the test case ID in "name" and in "tags" (e.g. "TC-1042").
Each manual step becomes one or more steps with clear labels; each
"Expected result" becomes an assertion step.
If a step cannot be automated reliably (CAPTCHA, email, printing), mark it
with a TODO- label and explain why.
<paste test cases from Excel / Jira / TestRail>

Worked example: example 4.

Follow-up prompts

“Check every target against the HTML I gave you. Replace any you invented with TODO-.”
“Which acceptance criteria have no assertion step? Add them.”
“atx validate reported: <paste output>. Fix the file.”
“The run failed at step <id> with <paste error from the run report>. Here is the current HTML: <paste>. Fix the target or the wait.”
“Make the status and total checks soft and add a full-page screenshot at the end.”

Getting the file into AutoTestX

Save the JSON as a file, for example po-create.json.
Validate it: atx validate po-create.json (schema, step types and expressions).
Optionally run it straight away: atx run po-create.json --env env.json --headed.
Import it into the project with POST /api/tests/import (see Importing a test file).
Open it in the editor, use Pick on any target marked TODO- or never used before, Run, then set the status to Ready.

4. Review checklist before saving any AI-written test

Every action in the requirement has a step, and every expected result has a check.
Checks compare exact text or values, not just “the page loaded”.
No real password, token or personal data anywhere; secret inputs are sensitive.
Targets prefer test id / role / label; no XPath, no generated CSS ids.
Each target matches exactly one element, and no target has {{…}} in a strategy value.
Every “is gone / is not visible” check follows a check that the element was there.
No fixed wait used to fix timing.
Records created by the test have unique names and are cleaned up.
The test passed in at least one Run and the report reads clearly.