Issues with Training "diffusion_autoencoder"

[vacuumisme]

I couldn't find any working configurations for training "diffusion_autoencoder", especially for the diffusion part. Also, the "dit" model seems too large to be practical as part of the diffusion autoencoder. I get CUDA out of memory even for small transformer and sample size = 65536

Does anybody train diffae and it really works?
Are there any working configs or examples?