Add SetAndCheckTenantSetting functional testing utility function #1448
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🗣 Description
Adds a new
SetAndCheckTenantSetting
to the functional testing utilities available for use in YAML-based testplans. The function supports taking two script blocks: one to change a tenant setting and another to test that the value has changed. If the value has not changed as expected, the set will be retried a user-defined number of times over a specified retry interval.In addition, both the SharePoint PnP and PowerPlatform test plans have been updated to use the retry functions in tests that have previously been problematic in terms of false positives due to configurations not changing as requested.
💭 Motivation and context
Attempts to resolve issues where a test pre-condition requires changing a tenant setting, but the setting doesn't reliably change or report actual success. The function is meant to ensure that the tenant reflects the requested configuration change prior to running the actual test.
Closes #1364
🧪 Testing
Either run one of the updated test plans or create/modify another testplan to use the function. It does require creating or migrating existing pre-condition commands into the SetBlock and creating a new CheckBlock that retrieves that value and checks its value, returning a boolean result.
✅ Pre-approval checklist
✅ Pre-merge checklist
PR passed smoke test check.
Feature branch has been rebased against changes from parent branch, as needed
Use
Rebase branch
button below or use this reference to rebase from the command line.Resolved all merge conflicts on branch
Notified merge coordinator that PR is ready for merge via comment mention
Demonstrate changes to the team for questions and comments.
(Note: Only required for issues of size
Medium
or larger)✅ Post-merge checklist