All nodes go into Donor/Desync after joining new node

[dbucher-datasport]

I currently had the same issue 2 times.

Setup:
Galera Manager 1.8.3 on host 1
Cluster with Ubuntu 20.04 and mysql 8
N1 synched
N2 synched
N3 synched

N1 is used from a application and has read/write actions
The database servers have about 80GB of data each

Issue reproduction:
Setting up a new node (n4) on ubuntu 20.04

Node setup runs until it trys to start the node.

Then n3 got into donor status, shortly after n2 gets into donor status and in the end n1 goes into donor status.

The log goes endless :
2024-01-26T16:36:55.449549Z 0 [Warning] [MY-000000] [WSREP] P: Member 0.0 (DSR-Core-n4-v2) requested state transfer from 'any', but it is impossible to select State Transfer donor: Resource temporarily unavailable

First time it happend I had to force delete the new node. All other nodes stayed in donor/desync. Had to shutdown all nodes and bootstrap the cluster to get it working again.

Second time i tried on a complete new installation of ubuntu 20.04 and got the same issuee.
I turned off the n4. Dint help
I restartet n3...now gives me the same error
2024-01-26T16:36:56.184421Z 0 [Warning] [MY-000000] [WSREP] P: Member 1.0 (DSR-Core-n3-hv) requested state transfer from 'any', but it is impossible to select State Transfer donor: Resource temporarily unavailable

I dint find any solutions online so far.

[dbucher-datasport]

I found that the 3 existing nodes run version 8.0.34-26.15 and the new created one 8.0.35-26.16...

Could this be related. Does the Manager not check the version of mysql at the existing nodes to install the same version?

[ayurchen]

No, this should not affect things at all. Could you please provide error logs from all nodes from the point where the new node joined the cluster and to the point where the third node became a DONOR.

[byte]

@dbucher-datasport is it possible to get error logs from all the nodes as asked for in the previous comment, thanks

[dbucher-datasport]

@ayurchen I've sent you the logs now in an email. Sorry for the late response.