Course: Self-healing — 4. How healing ranks candidates

Twelve hands-on exercises on tuning tier 1 self-healing: which signals it trusts, how sure it must be before a run acts on a healed element, and how to see what a change would have done before you save it (How healing ranks candidates):

Module 1: reading the ranking, and a preset per application;
Module 2: the project’s thresholds, and what they do to a run;
Module 3: your organization’s own profiles;
Module 4: minimum thresholds, and the rules they hold back;
Module 5: weights suggested from review decisions.

Each exercise has:

Goal: what you will see;
Before you start: the preparation it needs;
Steps: Action, Expected result, and a box to tick;
What happened: the explanation;
Check yourself: a question to answer before moving on. The answers are at the end.

Allow about 50 minutes. Do the exercises in order: each one starts from what the one before left.

For a complete check with the exact result of every step, use Healing profiles: a step-by-step check.

The two numbers behind every heal

When a step’s locators find nothing, healing scores each element of a kind the step can act on against the step’s last known good fingerprint:

Confidence is a weighted average of signals: what the element is known by (test hook, id, name, label, text…), what kind it is (role, tag…) and where it sits (heading, table row, position…). The weights say how much each signal counts.
Lead (margin) is how far the best candidate is ahead of the next one. A near tie is how a heal clicks the wrong one of two similar buttons.

A run acts on the best candidate only if its confidence reaches the bar for the step’s risk and its lead is big enough. Otherwise the heal is only suggested, for a person or a rule to decide. A preset or profile changes the weights; the project’s thresholds change the bars.

Preparation

P1: Two people

Who	Role	Window
You	Admin of the project and administrator of the organization	Window A
A colleague, called Bob below	Test Developer, organization Member	Window B, a private window

P2: An application, a test and one heal

In Window A:

Applications → New application, name Toolbar app, Save.
Test Cases → Import → test-plans/heal-conflict-toolbar.json. It is the Toolbar test: open a small page, click Save, click Cancel. The Open step’s URL picks a release of the page.
Run test with No environment: three green steps. This run records each button as its last known good element.
Click the Open step and change its URL from {{release1}} to {{release2}}. Ctrl+S. Release 2 gives the Save button a new id, class and text.
Run test again: Click Save is marked Healed. Leave the suggestion pending.

That one heal is the history the previews below replay.

Module 1: Reading the ranking

Exercise 1: Where the ranking lives

Goal: find what this project’s runs heal with today.

Before you start: P1, P2.

#	Action	Expected result
1	Window A: Self-healing → Rules	Under the daily limit, a card: a run acts at 70% on a low-risk step and 78% on a medium-risk one, with a lead of 12%; from 50% it suggests. “The defaults.”
2	Look below that	One row: Toolbar app, with the preset Balanced
3	Window B: the same page	The same card, but Balanced is plain text: no picker, no Change

What happened: the bars belong to the project, the weights to each application. Anyone in the project can read them; changing them takes permission to manage rules.

Check yourself: your project tests a Maximo screen and a React portal. Why would they want different weights but the same bars?

Exercise 2: Trying a preset without saving it

Goal: see what a preset would have done before you use it.

Before you start: Exercise 1.

#	Action	Expected result
1	Window A: open the Toolbar app picker	Balanced, Generated ids, Test hooks, Dense grids. Hover each to read when to use it
2	Pick Generated ids	Nothing is saved. A box: “Replayed 1 heal from the last 30 days with Generated ids for Toolbar app:”, then what changes
3	Read the box	Either “Every heal would have picked the same element.” or a red line naming a step that would have picked a different element. Then which decisions change
4	Click Keep Balanced	The box closes; nothing changed

What happened: the run kept the scores of the element it picked and of the next ones. The preview re-scores them under the preset’s weights, with no page and no browser. A different element is the line to read first: it means the preset would have fixed the test onto another control.

Check yourself: why does the preview replay heals with the current settings and the proposed ones, instead of comparing against what the run decided?

Module 2: The project’s thresholds

Exercise 3: Save waits for a preview

Goal: change the bars, seeing their effect first.

Before you start: Exercise 2.

#	Action	Expected result
1	In the ranking card, click Change → Set this project’s own	Four fields with their ranges and defaults. Save is greyed out: “Preview against history first”
2	Set the low-risk bar to `80`, Preview against history	“The medium-risk bar (78%) cannot be below the low-risk bar (80%).”
3	Set both bars to `99`, Preview against history	“1 acted on during the run → only suggested”. Save is enabled
4	Change the lead to `13`	The preview disappears and Save is greyed out again
5	Set the lead back to `12`, preview again, Save	“Saved this project’s thresholds”. The card shows 99% and “This project’s own thresholds.”

What happened: medium risk is never easier than low risk, so the pair is checked together. Any change after a preview makes it out of date, so Save waits for a new one.

Check yourself: the preview said the heal would be only suggested. Does that mean the fix would be lost?

Exercise 4: What a stricter bar does to a run

Goal: see the saved bars at work.

Before you start: Exercise 3.

#	Action	Expected result	✓
1	Open Toolbar, Run test	Click Save fails. Its error ends “below the 99% needed to act on a medium-risk step.”
2	Look at the Self-healing banner	The heal is still offered for review

What happened: the run found the same element at the same confidence, but 99% is more than it had, so it did not click it. A person (or a rule) can still approve the fix.

Check yourself: when would a team raise the bars, knowing more runs will fail?

Exercise 5: Back to the defaults

Goal: undo a threshold change.

Before you start: Exercise 4.

#	Action	Expected result	✓
1	Change → Use the defaults → Preview against history → Save	“Back to the default thresholds”
2	Run test on Toolbar	Click Save is Healed again

What happened: going back is a change like any other: previewed, saved, and recorded in the audit trail.

Check yourself: where would you look to find out who set the bars to 99% last week?

Module 3: Your organization’s own profiles

Exercise 6: A profile of your own, and the 50% rule

Goal: build a profile, and meet the one limit no profile can cross.

Before you start: Exercise 5. You are an organization administrator.

#	Action	Expected result
1	Organization → Projects	A card “Self-healing · how candidates are ranked”: no profiles yet, no minimums
2	New profile, name `Toolbar forms`	Three groups of weights. “What an element is known by carries 65% of the weight; at least 50% is required…”
3	Set Nearest heading, Ancestor ids, Table row and Position to `5`	The line turns red at 39% and Save is greyed out
4	Set them back to `1`, and Label to `2`	65%, and only Label is highlighted: it differs from Balanced
5	Save	“Toolbar forms v1 · based on Balanced”, “Not used yet”

What happened: a profile may trust anything you like, except that where an element sits (position, heading, row) can never outweigh what it is known by. Otherwise a deleted button’s neighbour, sitting where it used to be, would look like a good match.

Check yourself: an application’s ids change on every release. Which weights would you lower, and which would you raise?

Exercise 7: Using it

Goal: an organization profile is chosen like a preset, in every project.

Before you start: Exercise 6.

#	Action	Expected result	✓
1	Self-healing → Rules, open the Toolbar app picker	Two groups: “BUILT IN” and “YOUR ORGANIZATION” with Toolbar forms
2	Pick Toolbar forms, wait for the preview, Use Toolbar forms	“Toolbar app: Toolbar forms ranking”

What happened: the organization defines a profile once; each project decides which of its applications use it.

Check yourself: three projects test the same Maximo instance. What does a shared profile save you?

Exercise 8: Changing a profile that is in use

Goal: see why an edit to a shared profile is previewed across projects.

Before you start: Exercise 7.

#	Action	Expected result
1	Organization → Projects → Edit on Toolbar forms	“Used by Toolbar app (QA Project): saving changes how their next runs heal.” Save is greyed out
2	Set Label to `2.5`, Preview against history	“Replayed 1 heal from the last 30 days with these weights:”. Examples name the project and the step, not the test
3	Save	The card shows v2

What happened: the preview replays the heals of every application using the profile, in every project. You may not be a member of every project, so it names projects, never tests. Each report names the profile and version it was ranked with.

Check yourself: a heal last Tuesday ranked oddly. How do you find out which weights it used?

Exercise 9: A profile that cannot be deleted

Goal: see what protects applications from losing their profile.

Before you start: Exercise 8.

#	Action	Expected result	✓
1	Delete on Toolbar forms, confirm	‘“Toolbar forms” is used by Toolbar app (QA Project). Choose another profile for them first.’
2	Window B: Organization → Projects	Bob sees the profile, but cannot create, edit or delete one

What happened: deleting a profile in use would quietly change how those applications heal. The message names them, so you can move them first, with a preview each.

Check yourself: what does a run do if its application’s profile no longer exists (say, an older backup was restored)?

Module 4: Minimum thresholds

Exercise 10: An organization minimum

Goal: set a bar no project may go below.

Before you start: Exercise 9.

#	Action	Expected result
1	Organization → Projects → “Minimum thresholds” Change, medium-risk `90`, Save	“no project may act on a medium-risk step below 90%.”
2	Self-healing → Rules	The card says 90% for medium risk, and “The defaults, raised to your organization’s minimums.”
3	Change → Set this project’s own, medium-risk `85`, Preview against history	“The medium-risk bar cannot be below the organization’s minimum of 90%.” Cancel

What happened: a minimum holds every project up to it. A project that had set less keeps its own setting, held to the minimum, and gets it back if the minimum is lowered.

Check yourself: why is a minimum kept apart from the project’s own setting, instead of overwriting it?

Exercise 11: Rules and the minimum

Goal: see that a minimum binds rules too, and that a rule can always be switched off.

Before you start: Exercise 10.

#	Action	Expected result
1	New rule `Fix medium heals`: Apply automatically, risk medium only, minimum confidence `85`	“minConfidence 0.85 is below your organization’s minimum of 0.9 for automatic fixes. Raise it, or keep the rule disabled.”
2	Untick Enabled, Save	Saved, Disabled
3	Edit it: confidence `92`, Enabled, Save	Saved and enabled
4	Raise the medium minimum to `95` (as in Exercise 10), back to Rules	The rule says “Asks for 92%, below your organization’s minimum of 95%: fixes under the minimum wait for review. Raise it, or disable the rule.”
5	Disable it	It is disabled, and the warning is gone
6	Remove the minimum: clear the field, Save	“none - projects may use the full ranges.”

What happened: no rule fixes a test without review below the organization’s minimum. A rule saved before the minimum went up is held back by the guardrail when it runs, and its card says so. Switching a rule off is always allowed: it fixes nothing.

Check yourself: the rule in step 4 is still enabled. What happens to a 93% heal it matches?

Module 5: Weights from review decisions

Exercise 12: Asking the decisions

Goal: see what “Suggest from review decisions” needs before it proposes anything.

Before you start: Exercise 11.

#	Action	Expected result	✓
1	Organization → Projects → Edit on Toolbar forms → Suggest from review decisions	“Not enough to go on yet: 0 decisions, and it needs at least 20, with 5 approved and 5 rejected.” Cancel

What happened: every person who approves a heal says its element was right; every one who rejects or reverts it says, most often, that it was wrong. Once there are enough of both, the button compares how each signal scored on each side and proposes weights: a signal that tells right from wrong goes up, one that misleads goes down half as far. It never saves anything. The proposal is highlighted in the editor, and you preview it like any other change. A rule’s decisions do not count: a rule repeating its own conditions is not evidence.

Check yourself: why does a rejection count for less than an approval?

Answers

How their pages are built differs (Maximo generates ids, the portal may have test hooks), so they trust different signals. How much risk the team accepts when a run acts on its own is one decision for the project.
The run’s decision was made with whatever bars applied that day, and its scores are rounded. Replaying both sides compares like with like: unchanged settings show no change.
No. The run does not act on it, but the suggestion is filed as usual for a person or a rule to approve.
When a wrong click costs more than a failed run: payments, approvals, data that is hard to undo. A failed run with a suggestion is cheaper than a wrong action that passes.
Self-healing → Audit trail → Rule changes: Healing ranking changed, with the bars before and after and who changed them.
Lower Id (it is never the same twice); raise Label, Nearest heading and Table row, which survive a release. The Generated ids preset is a good start.
One set of weights to tune and review, instead of one copy per project drifting apart.
The heal’s report names the profile and version (and the project’s own bars, if any). The audit trail holds every version’s weights.
It falls back to Balanced, the default, rather than failing the run.
So that lowering or removing the minimum gives every project back what it chose, without anyone having to remember it.
It is held back by the guardrail and waits for a person: 93% is below the 95% minimum. The audit trail says why.
A rejection is not always a wrong element: sometimes the button was removed on purpose, or the fix was right but not wanted. An approval says more clearly that the element was the right one.