Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Add check for "rsyslogd: error" in /var/log/messages in all tests #388

Merged

Conversation

spetrosi
Copy link
Contributor

/var/log/messages sometimes contains errors not visible in journalctl.

Tests identified that some of our tests were resulting such errorsagainst certain platforms

@spetrosi
Copy link
Contributor Author

[citest]

@spetrosi spetrosi marked this pull request as draft May 27, 2024 18:39
@spetrosi
Copy link
Contributor Author

[citest]

@@ -364,7 +364,6 @@
service:
name: rsyslog
enabled: true
state: started
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What problem is being addressed by this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On latest Fedora this causes error messages in /var/log/messages as such:

May 25 09:43:31 7e33d79e-6b6e-4e22-97fc-fa84e46170bf rsyslogd: error during parsing file /etc/rsyslog.d/00-global.conf, on or before line 12: parameter 'workdirectory' specified more than once - one instance is ignored. Fix config [v8.2312.0-1.fc40 try https://www.rsyslog.com/e/2207 ]
May 25 09:43:31 7e33d79e-6b6e-4e22-97fc-fa84e46170bf rsyslogd: error during parsing file /etc/rsyslog.d/00-global.conf, on or before line 12: parameter 'workDirectory' not known -- typo in config file? [v8.2312.0-1.fc40 try https://www.rsyslog.com/e/2207 ]
May 25 09:43:31 7e33d79e-6b6e-4e22-97fc-fa84e46170bf rsyslogd: parameters for built-in module builtin:omfile already set - ignored  [v8.2312.0-1.fc40 try https://www.rsyslog.com/e/2220 ]
May 25 09:43:31 7e33d79e-6b6e-4e22-97fc-fa84e46170bf rsyslogd: module 'imuxsock' already in this config, cannot be added  [v8.2312.0-1.fc40 try https://www.rsyslog.com/e/2221 ]
May 25 09:43:31 7e33d79e-6b6e-4e22-97fc-fa84e46170bf rsyslogd: error during parsing file /etc/rsyslog.d/90-input-basics-basic_input.conf, on or before line 8: parameter 'SysSock.Use' not known -- typo in config file? [v8.2312.0-1.fc40 try https://www.rsyslog.com/e/2207 ]
May 25 09:43:31 7e33d79e-6b6e-4e22-97fc-fa84e46170bf rsyslogd: error during parsing file /etc/rsyslog.d/90-input-basics-basic_input.conf, on or before line 8: parameter 'SysSock.RateLimit.Interval' not known -- typo in config file? [v8.2312.0-1.fc40 try https://www.rsyslog.com/e/2207 ]

This is because the main config file was not generated yet. Why this does not fail on RHEL 8 and 9 - I have no idea.
But it seems like a good practice to start a service using a handler after the role invocation finishes.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On latest Fedora this causes error messages in /var/log/messages as such

Only on Fedora 40, but not 39, or on other platforms? Sounds like an issue with the fedora 40 rsyslog package.

This is because the main config file was not generated yet

The rsyslog package provides a default /etc/rsyslog.conf (and possibly files in /etc/rsyslog.d/), and the service should start with the default files. If this isn't working, it is a problem with the rsyslog package.

But it seems like a good practice to start a service using a handler after the role invocation finishes.

In general, yes, but in some cases the role needs the service to be running before it can take further actions.

changed_when: false
no_log: true

# /var/log/messages sometimes contains errors not visible in journalctl
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you give me an example of this happening? It seems like a journald configuration issue e.g. journal logs are being trimmed too quickly. Note that all log messages in /var/log/messages come from journal

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, tests_ovirt_elasticsearch.yml prints error messages only on EL 7. I fixed other tests in this PR and I am looking into this.

2024-05-29T11:46:14.984650-04:00 ibm-p8-kvm-03-guest-02 rsyslogd: error: 'tls.cacert' file /etc/rsyslog.d/es-ca.crt couldn't be accessed: No such file or directory  [v8.24.0-57.el7_9.3 try http://www.rsyslog.com/e/2039 ]
2024-05-29T11:46:14.984810-04:00 ibm-p8-kvm-03-guest-02 rsyslogd: error: 'tls.mycert' file /etc/rsyslog.d/es-cert.pem couldn't be accessed: No such file or directory  [v8.24.0-57.el7_9.3 try http://www.rsyslog.com/e/2039 ]
2024-05-29T11:46:14.984979-04:00 ibm-p8-kvm-03-guest-02 systemd: Started System Logging Service.
2024-05-29T11:46:14.985136-04:00 ibm-p8-kvm-03-guest-02 rsyslogd: error: 'tls.myprivkey' file /etc/rsyslog.d/es-key.pem couldn't be accessed: No such file or directory  [v8.24.0-57.el7_9.3 try http://www.rsyslog.com/e/2039 ]
2024-05-29T11:46:14.985322-04:00 ibm-p8-kvm-03-guest-02 rsyslogd: error: 'tls.cacert' file /etc/rsyslog.d/es-ca.crt couldn't be accessed: No such file or directory  [v8.24.0-57.el7_9.3 try http://www.rsyslog.com/e/2039 ]
2024-05-29T11:46:14.985492-04:00 ibm-p8-kvm-03-guest-02 rsyslogd: error: 'tls.mycert' file /etc/rsyslog.d/es-cert.pem couldn't be accessed: No such file or directory  [v8.24.0-57.el7_9.3 try http://www.rsyslog.com/e/2039 ]
2024-05-29T11:46:14.985633-04:00 ibm-p8-kvm-03-guest-02 rsyslogd: error: 'tls.myprivkey' file /etc/rsyslog.d/es-key.pem couldn't be accessed: No such file or directory  [v8.24.0-57.el7_9.3 try http://www.rsyslog.com/e/2039 ]

@spetrosi
Copy link
Contributor Author

[citest]

@spetrosi spetrosi force-pushed the fix-errors-in-varslogsmessages branch from 7f2a7ea to 0a30768 Compare June 3, 2024 10:11
@spetrosi
Copy link
Contributor Author

spetrosi commented Jun 3, 2024

[citest]

@spetrosi spetrosi force-pushed the fix-errors-in-varslogsmessages branch from 0a30768 to 4ed0583 Compare June 4, 2024 07:04
@spetrosi
Copy link
Contributor Author

spetrosi commented Jun 4, 2024

[citest]

@spetrosi
Copy link
Contributor Author

spetrosi commented Jun 4, 2024

[citest]

@spetrosi spetrosi marked this pull request as ready for review June 4, 2024 08:58
@richm richm merged commit 6e91b4a into linux-system-roles:main Jun 5, 2024
15 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants