You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Avocado has the test status WARN for "keeping track of noteworthy events that ultimately do not affect the test outcome" (see the description here), which is something sensible to me. However, there is a comment indicated that "some result plugins does not support this and report PASS instead" , which made me think about if we would better to make having the status or not configurable. Because, in case of we are going to present Avocado's test results to some result services, we may need to do the status replacement by having some external logic as the same reason. Therefore, I would like to request such a feature for conditionally dismissing all warnings by respecting users willingness.
Describe the solution you'd like
For users who do not want the WARN status to be presented in their test results we can introduce a new option (e.g. maybe job.run.dismiss_warnings?) for replacing all the WARN with PASS when the option was set, and that means they understand about the risk but just like to ignore it for a better communication experience with the downstream result services of their auto pipelines.
Describe alternatives you've considered
Maybe we could even let users define the test status model by providing the mapping of Avocado defined statuses to their own, for example:
Avocado defined statuses
User defined statuses
WARN
PASS
ERROR
FAIL
SKIP
(discard)
...
...
But this could be tricky in some way.
Additional information
The original issue was, after the fix of autotest/tp-qemu#3048, we found some tests report WARN instead of PASS due to we changed the logger of all the tp-qemu tests from the root logger to avocado.test and there is a particular rule (#444) for handling all the warning messages. We indeed can fix this issue by demoting all the warnings, but just consider if it would be better by doing in the other way around.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
Avocado has the test status
WARN
for "keeping track of noteworthy events that ultimately do not affect the test outcome" (see the description here), which is something sensible to me. However, there is a comment indicated that "some result plugins does not support this and reportPASS
instead" , which made me think about if we would better to make having the status or not configurable. Because, in case of we are going to present Avocado's test results to some result services, we may need to do the status replacement by having some external logic as the same reason. Therefore, I would like to request such a feature for conditionally dismissing all warnings by respecting users willingness.Describe the solution you'd like
For users who do not want the
WARN
status to be presented in their test results we can introduce a new option (e.g. maybejob.run.dismiss_warnings
?) for replacing all theWARN
withPASS
when the option was set, and that means they understand about the risk but just like to ignore it for a better communication experience with the downstream result services of their auto pipelines.Describe alternatives you've considered
Maybe we could even let users define the test status model by providing the mapping of Avocado defined statuses to their own, for example:
But this could be tricky in some way.
Additional information
The original issue was, after the fix of autotest/tp-qemu#3048, we found some tests report
WARN
instead ofPASS
due to we changed the logger of all the tp-qemu tests from the root logger toavocado.test
and there is a particular rule (#444) for handling all the warning messages. We indeed can fix this issue by demoting all the warnings, but just consider if it would be better by doing in the other way around.The text was updated successfully, but these errors were encountered: