Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conditionally dismiss all warnings per users configuration #5291

Open
luckyh opened this issue Mar 22, 2022 · 1 comment
Open

Conditionally dismiss all warnings per users configuration #5291

luckyh opened this issue Mar 22, 2022 · 1 comment

Comments

@luckyh
Copy link
Contributor

luckyh commented Mar 22, 2022

Is your feature request related to a problem? Please describe.
Avocado has the test status WARN for "keeping track of noteworthy events that ultimately do not affect the test outcome" (see the description here), which is something sensible to me. However, there is a comment indicated that "some result plugins does not support this and report PASS instead" , which made me think about if we would better to make having the status or not configurable. Because, in case of we are going to present Avocado's test results to some result services, we may need to do the status replacement by having some external logic as the same reason. Therefore, I would like to request such a feature for conditionally dismissing all warnings by respecting users willingness.

Describe the solution you'd like
For users who do not want the WARN status to be presented in their test results we can introduce a new option (e.g. maybe job.run.dismiss_warnings?) for replacing all the WARN with PASS when the option was set, and that means they understand about the risk but just like to ignore it for a better communication experience with the downstream result services of their auto pipelines.

Describe alternatives you've considered
Maybe we could even let users define the test status model by providing the mapping of Avocado defined statuses to their own, for example:

Avocado defined statuses User defined statuses
WARN PASS
ERROR FAIL
SKIP (discard)
... ...

But this could be tricky in some way.

Additional information
The original issue was, after the fix of autotest/tp-qemu#3048, we found some tests report WARN instead of PASS due to we changed the logger of all the tp-qemu tests from the root logger to avocado.test and there is a particular rule (#444) for handling all the warning messages. We indeed can fix this issue by demoting all the warnings, but just consider if it would be better by doing in the other way around.

@luckyh
Copy link
Contributor Author

luckyh commented Mar 22, 2022

@PaulYuuu @fbq815 feel free to add your comments here, thanks!

@richtja richtja moved this to Long Term (Next Q) Backlog in Default project Jan 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Long Term (Next Q) Backlog
Development

No branches or pull requests

1 participant