NXDOMAIN looking up TXT for _acme-challenge, even though the record can be found with google and cloudflare

[jakob42]

Hi,

just started using lego (cli, 4.14.0) and I have problems getting my certs validated via DNS. Sometimes it works, after trying 10 times. I upped the timeout to 15mins and still got those errors. Here is my latest run:

lego@sidley:~$ lego --accept-tos --email [email protected] --dns pdns --domains test.example.com  --dns.resolvers 1.1.1.1 --dns.resolvers 8.8.8.8 run
2023/08/23 09:15:48 [INFO] [test.example.com] acme: Obtaining bundled SAN certificate
2023/08/23 09:15:48 [INFO] [test.example.com] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/257544888216
2023/08/23 09:15:48 [INFO] [test.example.com] acme: Could not find solver for: tls-alpn-01
2023/08/23 09:15:48 [INFO] [test.example.com] acme: Could not find solver for: http-01
2023/08/23 09:15:48 [INFO] [test.example.com] acme: use dns-01 solver
2023/08/23 09:15:48 [INFO] [test.example.com] acme: Preparing to solve DNS-01
2023/08/23 09:15:48 [INFO] [test.example.com] acme: Trying to solve DNS-01
2023/08/23 09:15:48 [INFO] [test.example.com] acme: Checking DNS record propagation using [1.1.1.1:53 8.8.8.8:53]
2023/08/23 09:15:50 [INFO] Wait for propagation [timeout: 2m0s, interval: 2s]
2023/08/23 09:15:50 [INFO] [test.example.com] acme: Waiting for DNS record propagation.
[... more Waiting for DNS record propagation ...]
2023/08/23 09:16:49 [INFO] [test.example.com] acme: Waiting for DNS record propagation.
2023/08/23 09:16:58 [INFO] [test.example.com] acme: Cleaning DNS-01 challenge
2023/08/23 09:16:58 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/257544888216
2023/08/23 09:16:59 Could not obtain certificates:
        error: one or more domains had a problem:
[test.example.com] acme: error: 400 :: urn:ietf:params:acme:error:dns :: DNS problem: NXDOMAIN looking up TXT for _acme-challenge.test.example.com - check that a DNS record exists for this domain

During the command above, I used dig to check propagation of the entries. And they are instantly there, be it with the authorative NS, cloudflare or google:

``` root@sidley:~# dig -t TXT _acme-challenge.test.example.com @ns.example.com

; <<>> DiG 9.18.12-0ubuntu0.22.04.2-Ubuntu <<>> -t TXT _acme-challenge.test.example.com @ns.example.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 26796
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;_acme-challenge.test.example.com. IN TXT

;; ANSWER SECTION:
_acme-challenge.test.example.com. 120 IN TXT "VNpTtNGEuT227WlTvCyf14RaGBMLIpTHgNxysQ-8zNw"

;; Query time: 3 msec
;; SERVER: 134.102.137.37#53(ns.example.com) (UDP)
;; WHEN: Wed Aug 23 09:16:01 CEST 2023
;; MSG SIZE rcvd: 122

root@sidley:~# dig -t TXT _acme-challenge.test.example.com @1.1.1.1

; <<>> DiG 9.18.12-0ubuntu0.22.04.2-Ubuntu <<>> -t TXT _acme-challenge.test.example.com @1.1.1.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 2488
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;_acme-challenge.test.example.com. IN TXT

;; ANSWER SECTION:
_acme-challenge.test.example.com. 108 IN TXT "VNpTtNGEuT227WlTvCyf14RaGBMLIpTHgNxysQ-8zNw"

;; Query time: 11 msec
;; SERVER: 1.1.1.1#53(1.1.1.1) (UDP)
;; WHEN: Wed Aug 23 09:16:05 CEST 2023
;; MSG SIZE rcvd: 122

root@sidley:~# dig -t TXT _acme-challenge.test.example.com @8.8.8.8

; <<>> DiG 9.18.12-0ubuntu0.22.04.2-Ubuntu <<>> -t TXT _acme-challenge.test.example.com @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 48450
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;_acme-challenge.test.example.com. IN TXT

;; ANSWER SECTION:
_acme-challenge.test.example.com. 120 IN TXT "VNpTtNGEuT227WlTvCyf14RaGBMLIpTHgNxysQ-8zNw"

;; Query time: 23 msec
;; SERVER: 8.8.8.8#53(8.8.8.8) (UDP)
;; WHEN: Wed Aug 23 09:16:09 CEST 2023
;; MSG SIZE rcvd: 122

</details>

I'm baffled that it works sometimes and most of the times doesn't. I know DNS propagation can take its time, but I checked google and cloudflare and the appropriete txt records where there. Any idea what stupid mistake I'm doing? Is LetsEncrypt maybe asking a different DNS server, should I check another one?

Thanks!

[jakob42]

sometimes I'm also getting the following error, even though the three NS above returned the entry noted in the error message
[test.example.com] propagation: time limit exceeded: last error: NS ns.bigsss-bremen.de. did not return the expected TXT record [fqdn: _acme-challenge.test.example.com., value: 4wAjaB5SvOy9VoFwQYkBgZV3Yeu5rCy_b0RWiN0fd4c]:

[jakob42]

[test.example.com] propagation: time limit exceeded: last error: NS ns.example.com. did not return the expected TXT record [fqdn: _acme-challenge.test.example.com., value: q0u2cmqYvU2RQJPwsMWFy6Humm5ggT1eD7r2EFIWrao]:
I tested my NS, again it shows the expected result with dig from the same host.

Not sure what I might be doing wrong. There is no split DNS and Google and cloudflare "see" the correct entries as well