Course: Diagnosing failures — 5. Component maintenance

Ten hands-on exercises on the component maintenance queue: the list at the top of the Components page of shared components that need a person, with one recommendation each (Component maintenance queue):

Module 1: a component with no callers, and a healthy one;
Module 2: a component that fails in its callers, and why the failure class decides what to do: fix the component, or look at the application;
Module 3: a component that still passes, but only through a fallback locator, and the self-healing fix that cures it;
Module 4: housekeeping: unpublished changes and callers on an older version;
Module 5: the same queue for an AI assistant.

Each exercise has:

Goal: what you will see;
Before you start: the preparation it needs;
Steps: Action, Expected result, and a box to tick;
What happened: the explanation;
Check yourself: a question to answer before moving on. The answers are at the end.

Allow about 1 hour. Do the exercises in order: each one starts from what the one before left.

The six findings

Finding	Means	Recommendation
Failing	A component step failed in its callers’ latest runs	Depends on the failure class: fix and publish, or investigate the application
Fix waiting	Self-healing found a locator fix for a component step	Review it; approving publishes the next version
Drifting	A component step passed only through a fallback locator	Fix the locator before the fallback stops matching too
Unpublished	The source test changed since the latest version	Publish it once verified
Outdated callers	Callers are pinned to an older version	Review the impact and upgrade them
Unused	No test calls the component	Delete it if nothing will call it again

The queue is worked out fresh each time you open the Components page, from the latest run of each caller. Reload the page after every run to see the change.

Preparation

The exercises use a small sign-in page whose behaviour you switch. The browser that runs your tests has to reach it, so do these exercises on your own installation (Deployment): a hosted server cannot reach a page running on your computer.

In a PowerShell window in the AutoTestX folder:

node examples/component-maintenance/app.mjs

It prints sign-in demo on http://127.0.0.1:4440/login (mode ok). Leave it running. You switch the page by opening one of these addresses in a browser tab; each answers mode is now …:

Address	The sign-in page
`http://127.0.0.1:4440/mode?set=ok`	As recorded: a User field and a Sign in button `#sign-in`
`http://127.0.0.1:4440/mode?set=down`	Looks the same, but the sign-in service answers 503: you stay on the page
`http://127.0.0.1:4440/mode?set=gone`	Sign-in is closed for maintenance.: no field, no button
`http://127.0.0.1:4440/mode?set=drift`	The button’s id changed to `#signin-2026`; its text is still Sign in

P2: Start AutoTestX with its own data

In a second PowerShell window, so your normal data is not touched:

npm run build
npm run build:web
$env:ATX_DATA_DIR = ".data-components-training"; npm run start

Open the link printed after set the system administrator’s password, choose a password and sign in as the system administrator.

P3: Environment, suite and the source test

Configurations → Environments → New environment: name Sign-in demo, Variables (JSON) { "baseUrl": "http://127.0.0.1:4440" }, Save environment.
Test Suites → add a suite Login callers.
Test Cases → Import examples/component-maintenance/01-login-source.json. It is the test Login (component source), in suite Component sources, a Draft, with the test variable username = maxadmin. Its four steps: Open the sign-in page, Type the user name, Click Sign in (locators: #sign-in, then the text Sign in as a fallback) and Start Center is shown.
Open it, choose environment Sign-in demo and Run test. It passes.

Running the callers

From Exercise 2 on you run the three callers again and again. The quickest way: Test Suites → Run suites against = Sign-in demo → Run on Login callers, and wait for the panel to finish. Then open Testing → Components (reload it if it is open).

Module 1: No callers, and a healthy component

Exercise 1: An unused component

Goal: see the queue’s lowest finding, and create the component the rest of the exercises use.

Before you start: P1–P3.

#	Action	Expected result
1	Testing → Components → New component: Source test Login (component source), Name `Login`, Description `Signs in; ends on the Start Center.`	Parameters already lists `username`, default `maxadmin`
2	Create component	The component page: Login, version 1
3	Go back to Components	At the top, the card Maintenance queue with 1 to look at
4	Read the row	Unused · Login No test calls this component. · recommendation Delete it if nothing will call it again; its versions go with it. · button Open

What happened: a component nothing calls is housekeeping, at the bottom of any queue. It is listed so that forgotten components can be cleaned up, but a component you have only just created is unused too: the queue tells you, you decide.

Check yourself: why is deleting an unused component only a recommendation?

Exercise 2: A healthy component

Goal: see the queue empty itself once the component has callers that pass.

#	Action	Expected result
1	Test Cases → New test `Open work orders`, suite Login callers → Add → Call component `Login`, version 1 → Save	One step: the call to Login v1
2	Do the same for `Open assets` and `Open inventory`	Three callers
3	Run Login callers (see Running the callers)	PASS, 3 passed. Each run report shows the call, then Login’s four steps under it
4	Open Components	Maintenance queue nothing waiting: Every component passes in its callers’ latest runs, and every caller is on the latest version.
5	Look at the component list	Login · v1 · 3 tests · 3 calls

What happened: Unused disappeared as soon as a test called Login, and with three passing callers on the latest version there is nothing to do. The queue is not a log: a finding is there only while its cause is.

Check yourself: what has to be true of a component for the queue to say nothing waiting?

Module 2: A failing component

Exercise 3: The application is down

Goal: see a failure inside the component that is not the component’s fault.

#	Action	Expected result
1	Open `http://127.0.0.1:4440/mode?set=down`. Run Login callers	FAIL, 3 failed
2	In the panel, read the class line under a test	Server error 90% `POST 127.0.0.1:4440/api/session` (85% for the 503, +5% because it passed last time)
3	Open Components	One row: Failing · Login 1 step failed in the latest run of 3 callers (Server error).
4	Read the rest of the row	Steps: Start Center is shown, the first error (No strategy resolved…), and the recommendation The failure is classed Server error: the application, not the component, is the likely cause. Do not publish a new version for it. The three callers are named
5	Click Open run	The failed run’s report: the Login steps, and under Start Center is shown the 503 from `/api/session`
6	Open `http://127.0.0.1:4440/mode?set=ok`. Run Login callers, then open Components	PASS; the queue is back to nothing waiting

What happened: the step that failed is inside Login, so the queue traces three failing tests back to one component step and lists it once. But the sign-in service answered 503: the component did exactly what it should. Changing Login would not help, so the queue says to look at the application instead. That kind of finding also ranks lower than one the component’s owner can fix.

Check yourself: what would happen if you “fixed” Login by removing the Start Center is shown check?

Exercise 4: The page changed

Goal: see a failure the component’s owner has to fix.

#	Action	Expected result
1	Open `http://127.0.0.1:4440/mode?set=gone`. Run Login callers	FAIL, 3 failed; class line Locator 60%: 75% for the missing field, −15% because the callers have now passed and failed by turns (flaky)
2	Open Components	Failing · Login 1 step failed in the latest run of 3 callers (Locator). · Steps: Type the user name
3	Read the recommendation	Fix the step in “Login (component source)”, run it until it passes, then publish v2 and upgrade the 3 callers. · button Open
4	Open `http://127.0.0.1:4440/mode?set=ok`. Run Login callers, then open Components	PASS; nothing waiting

What happened: the sign-in form was gone, and nothing on the page could stand in for it, so self-healing offered no fix. A changed page is the component’s job: fix the step in the source test, publish a version, upgrade the callers. In a real project the page would stay changed; here you switched it back so the next exercise starts clean.

Check yourself: Exercises 3 and 4 both failed three callers at one component step. Which would you look at first if both happened at once, and why does the queue agree?

Module 3: Drifting, and the fix that cures it

Exercise 5: A fallback locator

Goal: catch a component that still passes, but will not for long.

#	Action	Expected result
1	Open `http://127.0.0.1:4440/mode?set=drift`. Run Login callers	PASS, 3 passed
2	Open a caller’s run report and look at Click Sign in	Passed with a warning: found by its fallback locator (the text Sign in), not `#sign-in`
3	Open Components	2 to look at, in this order: Fix waiting, then Drifting
4	Read Drifting	Login 1 step passed only through a fallback locator in 3 callers. · Steps: Click Sign in · Re-record or reorder the step’s locator in the source test and publish, before the fallback stops matching too.
5	Read Fix waiting	Login Self-healing found a locator fix for 1 step, best at 100% confidence (medium risk). · Approve the fix to publish v2, then upgrade the callers. · button Review fix

What happened: the button’s id changed, and only the fallback locator still found it. Every caller passed, so no dashboard turns red, but one more change to the page and all of them fail at once. Self-healing noticed too: it proposes promoting the locator that worked. Both are about the same step; the fix is listed first because it is the quickest cure.

Check yourself: why list a component that passes in every caller?

Exercise 6: Approve the fix, upgrade the callers

Goal: follow the recommendation through, and watch both findings go.

#	Action	Expected result
1	In the Fix waiting row, click Review fix	Self-healing: a pending fix for Click Sign in, in component Login v1
2	Approve it	The fix is approved; it publishes Login v2
3	Open Components	Fix waiting is gone. Drifting is still there, and a new row: Outdated callers · 3 callers still pin an older version than v2. · Review the impact and upgrade 3 tests to v2.
4	Click Review upgrade → on the component page Review upgrade to v2 (3) → tick all three → Upgrade 3 tests	Each caller now calls v2
5	Open Components	Only Drifting is left
6	Run Login callers (the page is still in drift)	PASS; Click Sign in passes with no warning now
7	Open Components	nothing waiting
8	Open `http://127.0.0.1:4440/mode?set=ok`	Back to normal

What happened: approving published v2 with the locator that works, but callers stay on the version they pin until someone upgrades them: that is what makes publishing safe. And Drifting is about the callers’ latest runs: they ran on v1, so it stays until they run again on v2. The queue only clears what has actually been proven.

Check yourself: after step 5 the callers are on v2. Why is Drifting still listed?

Module 4: Housekeeping

Exercise 7: Unpublished changes

Goal: notice a source test that has moved on without its component.

#	Action	Expected result
1	Open Login (component source). Change the label of step 4 from Start Center is shown to `Start Center is open` → Save	Saved
2	Open Components	Unpublished · Login “Login (component source)” has changed 1 step since v2 was published. · Steps: Start Center is open · If those changes are verified, publish them as v3; until then no caller sees them. · button Open source test
3	Open the component page → Publish new version → From test, What changed `Clearer check name` → Publish v3	Login v3
4	Open Components	Unpublished is gone. Outdated callers · 3 callers still pin an older version than v3. The component list shows 3 on an older version

What happened: the source test is where the next version is developed, so edits there are expected. But until they are published, no caller runs them: the queue reminds you so work does not sit unpublished. Publishing moves the finding on to the callers.

Check yourself: why does the queue compare the steps, not just the test’s version number?

Exercise 8: Outdated callers

Goal: finish the housekeeping.

#	Action	Expected result
1	Click Review upgrade → Review upgrade to v3 (3)	Upgrade callers of Login to v3: each caller with the version it is on (From) and What changes
2	Tick only `Open work orders` → Upgrade 1 test	—
3	Open Components	Outdated callers · 2 callers still pin an older version than v3.
4	Upgrade the other two	nothing waiting

What happened: you can upgrade callers one at a time, for example the ones you can run first. Outdated callers are the lowest-ranked finding after Unused: forty of them never outrank one failing step.

Check yourself: when would you leave a caller on an older version on purpose?

Module 5: Through the API and an AI assistant

Exercise 9: The queue as data

Goal: read the queue as a script would.

#	Action	Expected result	✓
1	Open `http://127.0.0.1:4440/mode?set=gone`, run Login callers	FAIL
2	Stay signed in and open `http://127.0.0.1:4100/api/component-maintenance`	A list with one item: `"kind":"failing"`, `"componentName":"Login"`, `"failureClass":"locator"`, a `priority`, the `summary`, `"recommendation":{"action":"fix-and-publish", …}`, `tests`, `steps` and `runIds`

What happened: the page and the API read the same queue, worked out on request. Nothing is stored, so it can never disagree with the runs.

Check yourself: what does priority let a script do that the summary does not?

Exercise 10: An AI assistant

Goal: see what an assistant gets when asked about components.

Before you start: Configurations → Project Members → API tokens for CI → create a token that can Run tests and copy its value. In a third PowerShell window:

npx @modelcontextprotocol/inspector

Set Transport Type Streamable HTTP, URL http://127.0.0.1:4100/api/mcp, and the header Authorization = Bearer + your token. Connect.

#	Action	Expected result
1	Tools → List Tools	`component_maintenance` is listed, read-only
2	Run component_maintenance with no arguments	The Login finding from Exercise 9, with its recommendation
3	Run it with `component` = `Nope`	Nothing to maintain in “Nope” (or no component has that name).
4	Open `http://127.0.0.1:4440/mode?set=ok`, run Login callers, then run the tool again	No component needs maintenance: every one passes in its callers’ latest runs and every caller is on the latest version.

What happened: an assistant asked “which components need maintenance?” gets the same queue and recommendations. It only reads: publishing, upgrading and approving stay with a person in AutoTestX.

Check yourself: twenty tests fail at sign-in. Why ask component_maintenance before explain_failure?

Clean up

Stop the windows (Ctrl+C) and delete the practice data:

Remove-Item -Recurse -Force .data-components-training

Answers

A component may be new, or kept for tests not written yet. Deleting also removes its versions, so a person decides.
Every caller’s latest run passes its component steps with their main locators, there is no fix waiting, the source test has not changed since the latest version, and every caller is on it.
The callers would “pass” while the sign-in still failed: a component without a check at the end spreads wrong passes into every test.
Exercise 4: the component’s owner can fix a changed page, while a down service waits for someone else. The queue ranks a failure classed as the application’s lower for that reason.
It is about to fail in every caller at once. Fixing it now, while it passes, costs one publish instead of a red suite.
Drifting comes from the callers’ latest runs, which ran on v1. It clears once they run on v2 and pass without the fallback.
Saving a Draft amends the same version of the test, so its number may not move even though its steps did.
When a caller cannot take the change yet: it is Ready and must stay as approved, or the new version needs a parameter it does not pass.
Sort or filter by urgency without reading text: for example, act only on findings above a threshold.
One component step can be behind all twenty failures: the queue finds it in one call, instead of twenty explanations of the same thing.