PDX-516: feat(mcp): validate UiAssert screen context (action=New vs View)#228
Merged
mrdailey99 merged 2 commits intoJun 24, 2026
Merged
Conversation
RCA: Create-and-verify tests that leave the UiAssert nested under the action=New create screen scored a clean 100 with no warning, because no validator checked whether a verification step sits in the correct post-save screen context. Fix: Added UI-SCREEN-CONTEXT-001 (check.type uiAssertScreenContext, MULTI_VALIDATOR_REGISTRY) with two heuristics over existing XML: UiAssert/UiRead nested under a UiWithScreen whose target URI carries action=New, and any UI action following a Save (locator binding action%3Dsave) within the same substeps block. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Quality Orchestrator🟢 LOW · 🧪 Tests to Run · Running 1 of 57 tests
▶ Run commandnpx vitest run \
unit/mcp/bestPracticesEngine.test.ts⚡ quality-orchestrator · |
RCA: The first cut of UI-SCREEN-CONTEXT-001 matched Save via raw substring (action%3dsave) so Save & New false-matched, ran heuristic 2 on every UiWithScreen so post-save steps under Edit/View false-fired, and emitted two violations for one step (assert after Save under New) double-penalising the local score (92.5 not 96.25). Fix: isSaveAction now parses the locator binding and compares the decoded action token exactly to save; heuristic 2 is scoped to action=New only; validateUiAssertScreenContext de-dups to at most one violation per offending testItemId. Added Edit and Save & New false-positive guard tests, locked the bad fixture to one violation and score 96.25, refreshed docs/notes, and bumped version 1.6.2 to 1.6.3. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
UI-SCREEN-CONTEXT-001(check.type: uiAssertScreenContext) to the best-practices validator, registered inMULTI_VALIDATOR_REGISTRYso it can emit one violation per offending step.major, weight5(consistent with the major runtime-anti-pattern tier).UiAssertnested under theaction=Newcreate screen scored a cleanis_valid:true / 100 / validwith no warning, while asserting against a page that no longer exists at runtime.Two heuristics (both over data already in the XML)
UiAssert/UiReadwhose nearest enclosingUiWithScreenhas a target URI withaction=New(the create screen). Salesforce tears down the create form on Save, so verifying there is the wrong context.UiWithScreen<clause name="substeps">block. Save is detected as aUiDoActionwhose locator binding containsaction=save(percent-encodedaction%3Dsavein the URI) and/orname=savewith anactioninteraction. Save navigates away, so trailing steps run in stale context.Both messages name the offending step (title + testItemId) and recommend moving it into a new
UiWithScreentargetingaction=View(or the appropriate post-save screen). The rule must NOT fire for the correct structure (assert under anaction=Viewscreen with nothing problematic after Save).Jira
https://provartesting.atlassian.net/browse/PDX-516
Test plan
opportunity-create-verify-bad.testcase— UiAssert moved into theaction=Newsubsteps after Save, standalone View screen removed) → rule fires; reported asmajor;quality_score < 100.opportunity-create-verify-good.testcase— assert underaction=View) → rule does NOT fire.action=View, nothing after Save → no violation from this rule.yarn compileclean; full mocha suite 1496 passing / 1 pending / 0 failing;yarn lintclean.node scripts/mcp-smoke.cjs30 PASS (4 pre-existing environmental timeouts on the globally-installed package; smoke does not exercise this rule).TOTAL_EXPECTEDunchanged — this PR adds a rule, not a tool.Changes
src/mcp/tools/bestPracticesEngine.ts— new validatorvalidateUiAssertScreenContext+ helpers (getUiWithScreenTargetUri,getScreenAction,getArgUri,isSaveAction,getScreenSubstepCalls,describeStep,flagAssertsOnNewScreen,flagStepsAfterSave); registered inMULTI_VALIDATOR_REGISTRY.src/mcp/rules/provar_best_practices_rules.json— new rule entryUI-SCREEN-CONTEXT-001.docs/mcp.md— documents the new rule, fix guidance, and a coverage-limit note under the runtime anti-pattern rules.docs/VALIDATION_RULE_REGISTRY.md— regenerated viascripts/build-validation-rule-registry.cjs(count 178 → 179, major 67 → 68).test/unit/mcp/bestPracticesEngine.test.ts— 5 new tests.test/fixtures/opportunity-create-verify-{good,bad}.testcase— negative + positive fixtures.Note for the Provar team
This adds a new customer-facing validator rule (
UI-SCREEN-CONTEXT-001) that changesprovar_testcase_validatebehaviour for create-and-verify Salesforce tests. The external/customer-facing docs (docs/provar-mcp-public-docs.md,docs/university-of-provar-mcp-course.md) are maintained separately and should be updated to describe the new rule.🤖 Generated with Claude Code