Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support loading configuration from both YAML files and env vars #831

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

lippserd
Copy link
Member

@lippserd lippserd commented Oct 26, 2024

I will keep the original PR description as a reference, but the implementation now uses config.Load() from our Go libs.

With this PR Icinga DB supports configuration loading in three scenarios:

  1. Load configuration solely from YAML files when no relevant environment variables are set.
  2. Combine YAML file and environment variable configurations, allowing environment variables to supplement or override possible incomplete YAML configurations.
  3. Load entirely from environment variables if the default YAML config file is absent and no specific config path is provided by the user.

config.FromYAMLFile() is still called first but continuation with config.FromEnv() is allowed by handling:

  • ErrInvalidConfiguration: The configuration may be incomplete and will be revalidated in config.FromEnv().
  • Non-existent file errors: If no explicit config path is set, fallback to environment variables is allowed.

config.FromEnv() is called regardless of the outcome from config.FromYAMLFile(). If no environment variables are set, configuration relies entirely on YAML. Otherwise, environment variables can supplement, override YAML settings, or serve as the sole source. config.FromEnv() also includes validation, ensuring completeness after considering both sources.

Possible alternative implementations:

  • If the config path is the default, os.Stat() could be used before calling config.FromYAMLFile(), rather than handling non-existent file errors. This approach would split the logic into two sections and add an additional if block.
  • Don't call Validate() in the config package automatically. Instead, require it to be called manually. I appreciate that library-wise, both config.FromYAMLFile() and config.FromEnv() include validation allowing them to be used without needing an additional function call on their own. When combining them, I think it's straightforward to use errors.Is() to check for ErrInvalidConfiguration, i.e. errors from Validate().

requires Icinga/icinga-go-library#87
requires #879

closes #756

@cla-bot cla-bot bot added the cla/signed label Oct 26, 2024
@lippserd lippserd marked this pull request as ready for review October 28, 2024 11:52
@lippserd lippserd force-pushed the config-from-environment-variables branch 2 times, most recently from 65232f7 to f73a444 Compare October 31, 2024 10:58
@lippserd lippserd added this to the 1.3.0 milestone Dec 5, 2024
dependabot bot and others added 2 commits January 7, 2025 18:30
Bumps [github.com/icinga/icinga-go-library](https://github.com/icinga/icinga-go-library) from 0.4.0 to 0.5.0.
- [Commits](Icinga/icinga-go-library@v0.4.0...v0.5.0)

---
updated-dependencies:
- dependency-name: github.com/icinga/icinga-go-library
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
internal/config/config.go Show resolved Hide resolved
internal/config/config.go Show resolved Hide resolved
internal/config/config.go Show resolved Hide resolved
@lippserd lippserd force-pushed the config-from-environment-variables branch from 8f5e40e to b18e594 Compare January 15, 2025 15:11
@lippserd lippserd requested a review from oxzi January 15, 2025 15:11
if err := config.Load(&cfg, config.LoadOptions{
Flags: flags,
EnvOptions: config.EnvOptions{Prefix: "ICINGADB_"},
}); err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works as far as I have tested it.

However, the error output may be misleading for a non-existing /etc/icingadb/config.yml file when not explicitly setting the --config flag, relying on the default.

$ stat /etc/icingadb/config.yml
stat: cannot statx '/etc/icingadb/config.yml': No such file or directory

$ ./icingadb
invalid configuration: database host missing
$ ./icingadb -c /etc/icingadb/config.yml
can't open YAML file /etc/icingadb/config.yml: open /etc/icingadb/config.yml: no such file or directory

If there are no ICINGADB_ environment variables and the default config is not existing, the reported error does not state that the expected file does not exist, but says that certain fields are absent. But honestly, I don't know how to address this. For a package installations the config file should be present.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is also the config_test.go file, testing against the now unused config.FromYAMLFile function. Maybe you want to remove this test case and create another one for YAML-only, environment-only, and mixed-setups.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants