fixrule(meta_refresh_delay | meta_redirect_optional) Reports "cannot tell" and "passed" while ACT bisz58 expects "Failed" for 3 Fail Example test cases

[philljenkins]

Checker rule meta_redirect_optional is also involved with these ACT rule(s) below.

Checker rule meta_refresh_delay reports inconsistent with ACT rule(s) below:

• Failed examples should be failed by at least one of the Checker rules fail reason codes.
• Checker should be mapping (report against) the same success criteria as what is expected by the ACT rule.

4. Partially inconsistent with ACT rule Meta element has no refresh delay (no exception) bisz58 with 3 failed test cases:
   • "cannot tell" Failed Example 1
   • "passed" Failed Example 2
   • "passed" Failed Example 3
   • Mapping: Checker reports against 2.2.1 Timing Adjustable, while ACT expects both 2.2.4 Interruptions AAA and 3.2.5 Change on Request AAA.
   • RuleID: meta_refresh_delay with ReasonID: Potential_1 reports against bisz58:cantTell | bc659a:cantTell
   • RuleID: meta_refresh_delay with ReasonID: Pass_0 also reports against bisz58:pass | bc659a:pass
   • RuleID: meta_redirect_optionalwith ReasonID: fail_longrefresh flags (doesn't report to ACT) this Fail Example 2 test case as Violation, but it's mapped to 2.2.1 Timing Adjustable while ACT would expect it to be mapped to 2.2.4 Interruptions AAA and 3.2.5: Change on Request AAA. Again, this ...optional rule does not currently report against this ACT bisz58. The ...optional rule currently only report against ACT bc659a.
   • Secondary requirement: ACT considers 2.2.1 Timing Adjustable (Level A) a "secondary requirement" because 2.2.1 is less strict than this ACT bisz58 rule. 2.2.1 allows redirects longer than 20 hours. The meta element in Failed Example 2 redirects after 20 hours.

Reference

• Checker's Implementation report of ACT rules https://www.w3.org/WAI/standards-guidelines/act/implementations/equal-access/ (updated 7 November 2024)
• Checker's raw results sent to ACT https://unpkg.com/[email protected]/act-report-v2.txt
• Checker's Rule mapping spreadsheet file on box

[philljenkins]

See latest Rule mapping spreadsheet:

[philljenkins]

Relevant Artifacts

• however, these are auto-built with master 3.x and include the reporting failures. I will need to replace them with the master 4.x artifacts:

master 3.x artifacts

• ACT Text Results
• Rule Listing
• Rule mapping spreadsheet

master 4.x artifacts

• ACT Text Results
• Rule listing
• Rule mapping spreadsheet

[philljenkins]

Ignoring these test cases in ACT mapping/reporting. We test/QA to be concerned about fail test cases not failed by Checker and correct WCAG SC reporting in this main 4.x Artifact ACT Text Results 2025-01-14 from PR #2158 :

Ignore

• 6a7281 Passed Example 5: Expected passed, but returned earl:failed
• akn7bn Inapplicable Example 5: Expected inapplicable, but returned earl:failed
• akn7bn Inapplicable Example 6: Expected inapplicable, but returned earl:failed
• 73f2c2 Passed Example 8: Expected passed, but returned earl:failed
• 7d6734 Passed Example 2: Expected passed, but returned earl:failed
• etc.

[philljenkins]

Expected Failed

These are somewhat of a surprises in 4.x. These ACT "fail" test cases should be reported as failed, but were not in Checker 4.x Artifact ACT Text Results 2025-01-14 from PR #2158

Although this is not completed, if these are not resolved they would results in more inconsistencies than previously reported:

ACT Rule 23a2a8 Image has non-empty accessible name:

• 23a2a8 Failed Example 2: Expected failed, but returned earl:inapplicable. Checker should fail this test case.

ACT Rule 7d6734 SVG element with explicit role has non-empty accessible name:

• 7d6734 Failed Example 4: Expected failed, but returned earl:inapplicable. Checker should fail this test case.

ACT Rule b4f0c3 Meta viewport allows for zoom:

• b4f0c3 Failed Example 1: Expected failed, but returned earl:inapplicable. Checker should fail this test case.
  • matches URL at act/rules/b4f0c3/#test-cases accc6adf094723693593ca3c6308f81945930dae.html

These next two appear to be the same test case "Failed Example 2", but have different URLss.

• b4f0c3 Failed Example 2: Expected failed, but returned earl:inapplicable. Checker should fail this test case. Note URL a1240b31761f65c92a8f6d08ed7105ee822d0ebc.html
  • matches URL at act/rules/b4f0c3/#test-cases a1240b31761f65c92a8f6d08ed7105ee822d0ebc.html
• b4f0c3 Failed Example 2 Expected failed, but returned earl:inapplicable. Checker should fail this test case. Note URL 8735b02e59dd802a5a9e9a7ea1934a106d2963eb.html
  • does not match URL at at act/rules/b4f0c3/#test-cases
• b4f0c3 Failed Example 3 Expected failed, but returned earl:inapplicable. Checker should fail this test case. Note URL c7e4980eb42d0b138684ac1e28318a197d86ccfc.html
• b4f0c3 Failed Example 3 Expected failed, but returned earl:inapplicable. Checker should fail this test case. Note URL c94a59f8c3b17d722781af36da3556ff4b418776.html WAR The value "invalid" for key "user-scalable" is invalid, and has been ignored.
• b4f0c3 Failed Example 4 Expected failed, but returned earl:inapplicable. Checker should fail this test case.