feat: add option to use only new format TXT records #4946

malpou · 2024-12-12T21:06:47Z

Description

Adds a new flag --txt-new-format-only that allows configuring external-dns to only create TXT records in the new format, containing record type information. This helps reduce the number of DNS records created when using external-dns at scale, addressing provider-specific record limits.

By default, the TXT registry creates both old and new format records for backwards compatibility. The new format includes the record type in the name (e.g., "a-" prefix for A records). With this change, users can opt-in to using only the new format when all their external-dns instances support it.

Key changes:

Add --txt-new-format-only flag (defaults to false for backwards compatibility)
Add configuration option to TXTRegistry
Update TXT record generation to respect the new setting
Add tests for the feature
Maintain existing behavior by default

Example usage:

# Default behavior - creates both old and new format records
external-dns

# Enable new format only - reduces number of TXT records
external-dns --txt-new-format-only

Note: AAAA records already only create new format records regardless of this setting, as they have special handling.

Fixes #3167
Fixes #4636

Checklist

Unit tests updated
End user documentation updated

linux-foundation-easycla · 2024-12-12T21:06:52Z

The committers listed above are authorized under a signed CLA.

✅ login: malpou / name: Malthe Poulsen (c7154c9, 7c22a03, 0d56712, ab3dfe6, fd9091a, 64a1b4d, 438d565, 8937026, 4c2e124)

k8s-ci-robot · 2024-12-12T21:06:54Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign mloiseleur for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot · 2024-12-12T21:06:57Z

Welcome @malpou!

It looks like this is your first PR to kubernetes-sigs/external-dns 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/external-dns has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

k8s-ci-robot · 2024-12-12T21:06:58Z

Hi @malpou. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

ivankatliarchuk · 2025-01-04T12:51:01Z

docs/registry/txt.md

@@ -3,6 +3,24 @@
 The TXT registry is the default registry.
 It stores DNS record metadata in TXT records, using the same provider.

+## Record Format Options


Could you include a subsection on how to handle the migration from two records to a single record? Specifically, what are the steps involved in this consolidation, and will the external DNS automatically delete the old records or will manual cleanup be necessary?

Right now i haven't thought of any automatic cleanup, so for now it would be a manual cleanup for the old format TXT records.

Is that an okay approach, or would you like it to happen automatically?

Mentioning this in the documentation likely is enough.

Added an extra section describing a manual migration flow.

ivankatliarchuk · 2025-01-04T12:51:50Z

docs/registry/txt.md

+external-dns
+
+# Only create new format records
+external-dns --txt-new-format-only


Is this required - --managed-record-types=TXT or with that flags we imply implicitly that TXT records should be handled?

The flag, is the initial implementation thought as an addition to the flags you would already start external-dns with.

So no it doesn't implicitly tell that we are using TXT records.

The usage of the flag should be more clear now.

ivankatliarchuk · 2025-01-04T12:58:09Z

resolves #3420

relates #3757 I do not think this issue is a blocker, as we could deprecate flags or plan migrations

pr-refresh #3744

pr that potentially need to be merged first #3774

ivankatliarchuk · 2025-01-04T12:59:18Z

docs/registry/txt.md

+- Legacy format: Creates a TXT record without record type information
+- New format: Creates a TXT record with record type information (e.g., 'a-' prefix for A records)
+
+By default, the TXT registry creates records in both formats for backwards compatibility. You can configure it to use only the new format by using the `--txt-new-format-only` flag. This reduces the number of TXT records created, which can be helpful when working with provider-specific record limits.


Can we add a bit more information about this behaviour as well? e.g. why external-dns creates two records

I don't have the history of why there are 2 formats, but would be glad to add some more text about it.

We just reached a point in my team at work where we would like to get rid of our duplicate TXT records in all of our DNS zones.

ivankatliarchuk · 2025-01-04T13:03:30Z

pkg/apis/externaldns/types.go

@@ -138,6 +138,7 @@ type Config struct {
 	TXTSuffix                          string
 	TXTEncryptEnabled                  bool
 	TXTEncryptAESKey                   string `secure:"yes"`
+	TXTNewFormatOnly                   bool


would it be possible to update types_test.go as well?

Added tests for the new flag now

ivankatliarchuk · 2025-01-04T13:52:42Z

Is this flag --txt-new-format-only could be considered like always use default latest/new format?

malpou · 2025-01-04T14:18:19Z

I'd like for the new format to be the default thing, but i don't have enough insight to know if that's currently possible??

This PR, focuses on giving people the option to move over to the new format without breaking anything for people that don't want that at this point in time.

It seems like there are multiple persons that would like this option (myself included), without having to move over to encrypted records to get rid of the extra records.

malpou added 2 commits December 12, 2024 20:33

feat: add option to use only new format TXT records

0d56712

add flag and docs

c7154c9

k8s-ci-robot requested review from mloiseleur and szuecs December 12, 2024 21:06

k8s-ci-robot added the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label Dec 12, 2024

k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Dec 12, 2024

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Dec 12, 2024

ivankatliarchuk reviewed Jan 4, 2025

View reviewed changes

z0rc mentioned this pull request Jan 4, 2025

Option to use only new TXT format #2884

Open

malpou added 6 commits January 4, 2025 15:36

Merge branch 'master' into feat/txt-registry-new-format-only

7c22a03

refine documentation on how to use the flag

fd9091a

add section regarding manual migration

64a1b4d

update documentation to be same as in types.go

4c2e124

fix compile issue

438d565

add tests for new flag

8937026

k8s-ci-robot added cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. and removed cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jan 4, 2025

malpou force-pushed the feat/txt-registry-new-format-only branch from 7ad3deb to 8937026 Compare January 4, 2025 15:03

k8s-ci-robot removed the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label Jan 4, 2025

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jan 4, 2025

update flags documentation correctly

ab3dfe6

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add option to use only new format TXT records #4946

feat: add option to use only new format TXT records #4946

malpou commented Dec 12, 2024 •

edited

Loading

linux-foundation-easycla bot commented Dec 12, 2024 •

edited

Loading

k8s-ci-robot commented Dec 12, 2024

k8s-ci-robot commented Dec 12, 2024

k8s-ci-robot commented Dec 12, 2024

ivankatliarchuk Jan 4, 2025

malpou Jan 4, 2025

ivankatliarchuk Jan 4, 2025

malpou Jan 4, 2025

ivankatliarchuk Jan 4, 2025

malpou Jan 4, 2025

malpou Jan 4, 2025

ivankatliarchuk commented Jan 4, 2025 •

edited

Loading

ivankatliarchuk Jan 4, 2025

malpou Jan 4, 2025

ivankatliarchuk Jan 4, 2025

malpou Jan 4, 2025

ivankatliarchuk commented Jan 4, 2025 •

edited

Loading

malpou commented Jan 4, 2025 •

edited

Loading

feat: add option to use only new format TXT records #4946

Are you sure you want to change the base?

feat: add option to use only new format TXT records #4946

Conversation

malpou commented Dec 12, 2024 • edited Loading

linux-foundation-easycla bot commented Dec 12, 2024 • edited Loading

k8s-ci-robot commented Dec 12, 2024

k8s-ci-robot commented Dec 12, 2024

k8s-ci-robot commented Dec 12, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ivankatliarchuk commented Jan 4, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ivankatliarchuk commented Jan 4, 2025 • edited Loading

malpou commented Jan 4, 2025 • edited Loading

malpou commented Dec 12, 2024 •

edited

Loading

linux-foundation-easycla bot commented Dec 12, 2024 •

edited

Loading

ivankatliarchuk commented Jan 4, 2025 •

edited

Loading

ivankatliarchuk commented Jan 4, 2025 •

edited

Loading

malpou commented Jan 4, 2025 •

edited

Loading