Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DAOS-16220 test: Include username in default common test directory #14937

Draft
wants to merge 63 commits into
base: master
Choose a base branch
from
Draft

DAOS-16220 test: Include username in default common test directory #14937

wants to merge 63 commits into from

Conversation

phender
Copy link
Contributor

@phender phender commented Aug 15, 2024

To support multiple users testing in the same environment without sudo access the default common test directory will be made unique by including the user ID in the path.

Skip-unit-tests: true
Skip-fault-injection-test: true

Required-githooks: true

Before requesting gatekeeper:

  • Two review approvals and any prior change requests have been resolved.
  • Testing is complete and all tests passed or there is a reason documented in the PR why it should be force landed and forced-landing tag is set.
  • Features: (or Test-tag*) commit pragma was used or there is a reason documented that there are no appropriate tags for this PR.
  • Commit messages follows the guidelines outlined here.
  • Any tests skipped by the ticket being addressed have been run and passed in the PR.

Gatekeeper:

  • You are the appropriate gatekeeper to be landing the patch.
  • The PR has 2 reviews by people familiar with the code, including appropriate owners.
  • Githooks were used. If not, request that user install them and check copyright dates.
  • Checkpatch issues are resolved. Pay particular attention to ones that will show up on future PRs.
  • All builds have passed. Check non-required builds for any new compiler warnings.
  • Sufficient testing is done. Check feature pragmas and test tags and that tests skipped for the ticket are run and now pass with the changes.
  • If applicable, the PR has addressed any potential version compatibility issues.
  • Check the target branch. If it is master branch, should the PR go to a feature branch? If it is a release branch, does it have merge approval in the JIRA ticket.
  • Extra checks if forced landing is requested
    • Review comments are sufficiently resolved, particularly by prior reviewers that requested changes.
    • No new NLT or valgrind warnings. Check the classic view.
    • Quick-build or Quick-functional is not used.
  • Fix the commit message upon landing. Check the standard here. Edit it to create a single commit. If necessary, ask submitter for a new summary.

@phender
DAOS-16220 test: Include username in default common test directory
995b03b 
To support multiple users testing in the same environment without sudo
access the default common test directory will be made unique by
including the user ID in the path.

Skip-unit-tests: true
Skip-fault-injection-test: true

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 15, 2024 •
edited
Loading

Ticket title is 'ftest: Include username in default common test directory'
Status is 'Awaiting backport'
Labels: 'daos_framework,testp2'
Job should run at elevated priority (1)
https://daosio.atlassian.net/browse/DAOS-16220

phender added 2 commits August 15, 2024 17:57
@phender
Merge branch 'master' into pahender/DAOS-16220
3fc3b4f 
Skip-unit-tests: true
Skip-fault-injection-test: true

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@phender
Fix pylint.
f0b01e9 
Skip-unit-tests: true
Skip-fault-injection-test: true

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
phender added 5 commits August 28, 2024 18:19
@phender
Merge branch 'master' into pahender/DAOS-16220
7907b85 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: test_setup_vm test_setup
Skip-func-hw-test-medium-verbs-provider: true

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@phender
Fix shellcheck and apply feedback.
5b031ec 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: test_setup_vm test_setup
Skip-func-hw-test-medium-verbs-provider: true

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@phender
Fix main.sh
bb345cb 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: test_setup_vm test_setup
Skip-func-hw-test-medium-verbs-provider: true

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@phender
Fix main.sh again
bb3ef4c 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: test_setup_vm test_setup
Skip-func-hw-test-medium-verbs-provider: true

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@phender
Test custom_a mode
c521baa 
Skip-unit-tests: true
Skip-fault-injection-test: true

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Medium completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-14937/7/display/redirect

@daosbuild1
Copy link
Collaborator

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14937/8/execution/node/824/log

@phender
Fix systemctl --user usage.
09fbcfb 
Skip-unit-tests: true
Skip-fault-injection-test: true

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@daosbuild1
Copy link
Collaborator

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14937/9/execution/node/823/log

@phender
Enable user systemctl on all test nodes.
681b8cc 
Skip-unit-tests: true
Skip-fault-injection-test: true

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@daosbuild1
Copy link
Collaborator

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14937/10/execution/node/823/log

phender added 2 commits August 29, 2024 23:17
@phender
Merge branch 'master' into pahender/DAOS-16220
8d54966
@phender
Fixing running agent as user.
2ac57c5 
Test-tag: test_setup_vm
Skip-unit-tests: true
Skip-fault-injection-test: true

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@daosbuild1
Copy link
Collaborator

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14937/11/execution/node/823/log

@phender
Cleanup verify socket dir.
bfeb3b3 
Test-tag: test_setup_vm
Skip-unit-tests: true
Skip-fault-injection-test: true

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@daosbuild1
Copy link
Collaborator

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14937/12/execution/node/823/log

phender added 3 commits August 30, 2024 14:26
@phender
Cleanup verify socket dir.
810c456 
Test-tag: test_setup_vm
Skip-unit-tests: true
Skip-fault-injection-test: true

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@phender
Remove missing config temp error.
1908088 
Test-tag: test_setup_vm
Skip-unit-tests: true
Skip-fault-injection-test: true

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@phender
Merge branch 'pahender/DAOS-16220' of https://github.com/daos-stack/daos
3bb43b6 
 into pahender/DAOS-16220

Test-tag: test_setup_vm
Skip-unit-tests: true
Skip-fault-injection-test: true

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@daosbuild1
Copy link
Collaborator

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14937/13/execution/node/798/log

phender added 2 commits August 30, 2024 16:04
@phender
Merge branch 'master' into pahender/DAOS-16220
4565846
@phender
Fix running systemctl as user.
f89984a 
Test-tag: test_setup_vm
Skip-unit-tests: true
Skip-fault-injection-test: true

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@daosbuild1
Copy link
Collaborator

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14937/14/execution/node/823/log

@phender
Adding systemctl status for failed start debug.
1df9183 
Test-tag: test_setup_vm
Skip-unit-tests: true
Skip-fault-injection-test: true

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@daosbuild1
Copy link
Collaborator

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14937/15/execution/node/798/log

phender added 2 commits October 3, 2024 13:57
@phender
Merge branch 'master' into pahender/DAOS-16220
282d39c
@phender
Add user to stop_processes.
48fcdb1 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: pr control test_agent_failure_isolation

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
daltonbohning
daltonbohning reviewed Oct 4, 2024
View reviewed changes
src/tests/ftest/util/job_manager_utils.py Outdated
str: regular expression to use to search for the command

"""
return "'({})'".format("|".join(self._exe_names + self.job._exe_names))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

protected-access, Access to a protected member _exe_names of a client class

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed, will address in DAOS-16100

src/tests/ftest/util/server_utils.py
@@ -45,7 +46,7 @@ def get_server_command(group, cert_dir, bin_dir, config_file, config_temp=None):
transport_config = DaosServerTransportCredentials(cert_dir)
common_config = CommonConfig(group, transport_config)
config = DaosServerYamlParameters(config_file, common_config)
command = DaosServerCommand(bin_dir, config, None)
command = DaosServerCommand(bin_dir, config, None, run_user)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a big deal, but the server fundamentally cannot be ran as a non-privileged user so this will probably never be used

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood, this is more to keep the code aligned and to avoid any special handling where an attribute may not be set.

phender added 2 commits October 4, 2024 17:17
@phender
Merge branch 'master' into pahender/DAOS-16220
50d5422
@phender
Apply feedback.
0a2cf6b 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: pr control test_agent_failure_isolation daily_regression

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@github-actions github-actions bot added the priority Ticket has high priority (automatically managed) label Oct 4, 2024
@daosbuild1
Copy link
Collaborator

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14937/42/execution/node/800/log

@phender
Fix handling daos_user home dirs.
5109544 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: pr control test_agent_failure_isolation daily_regression

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@daosbuild1
Copy link
Collaborator

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14937/43/execution/node/825/log

phender added 2 commits October 8, 2024 13:50
@phender
Merge branch 'master' into pahender/DAOS-16220
3776f23
@phender
Fix typo.
75d8677 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: pr control test_agent_failure_isolation daily_regression

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Medium Verbs Provider completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14937/44/execution/node/1158/log

@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Medium completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14937/44/execution/node/1112/log

@phender
Copy link
Contributor Author

phender commented Oct 14, 2024

Failures in https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-14937/44/testReport/ are due to known issues:

@phender
Merge branch 'master' into pahender/DAOS-16220
b28ae35
@phender
Merge branch 'master' into pahender/DAOS-16220
12d0c0e 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: pr control test_agent_failure_isolation daily_regression

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Medium Verbs Provider completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14937/45/execution/node/1116/log

@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Medium completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14937/45/execution/node/1100/log

@phender phender marked this pull request as ready for review October 22, 2024 20:26
@phender phender requested a review from daltonbohning October 22, 2024 20:26
@phender phender marked this pull request as draft October 22, 2024 20:33
daltonbohning
daltonbohning previously approved these changes Oct 22, 2024
View reviewed changes
@phender
Setting DAOS_AGENT_DRPC_DIR for remote commands
2507648 
Adding SubprocessManager.get_journalctl() method to ensure command is
run with the correct user.

Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: pr control test_agent_failure_isolation daily_regression AgentFailure CriticalIntegrationWithServers CriticalIntegrationWithoutServers ContinuesAfterStop TestWithScrubberTargetEviction soak_smoke

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
@phender
Fix pylint issue.
a4ff69c 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: pr control test_agent_failure_isolation daily_regression AgentFailure CriticalIntegrationWithServers CriticalIntegrationWithoutServers ContinuesAfterStop TestWithScrubberTargetEviction soak_smoke

Required-githooks: true

Signed-off-by: Phil Henderson <[email protected]>
phender
phender commented Oct 22, 2024
View reviewed changes
src/tests/ftest/util/general_utils.py
@@ -444,7 +448,7 @@ def get_host_data(hosts, command, text, error, timeout=None):
data_error = "[ERROR]"

# Find the data for each specified servers
log.info(" Obtaining %s data on %s", text, hosts)
log.info(" Obtaining %s data on %s using: %s", text, hosts, command)
Copy link
Contributor Author

@phender phender Oct 22, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to compensate for the loss of logging the journalctl command in src/tests/ftest/util/soak_utils.py old line 224.

@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Large completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14937/47/execution/node/1025/log

@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Medium Verbs Provider completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14937/47/execution/node/1169/log

@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Medium completed with status UNSTABLE. https://build.hpdd.intel.com/job/daos-stack/job/daos//view/change-requests/job/PR-14937/47/testReport/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@daltonbohning daltonbohning daltonbohning left review comments

At least 2 approving reviews are required to merge this pull request.

Assignees
No one assigned
Labels
priority Ticket has high priority (automatically managed)
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@phender @daosbuild1 @daltonbohning