Megatron Support

[huyiwen]

🚀 The feature, motivation and pitch

Thank you for developing such an outstanding library. With its help, my pre-training work has achieved an MFU of 51%. Truly appreciate your library.

In larger-scale training scenarios, frameworks based on Megatron are commonly utilized for training. I was wondering if it would be possible to integrate Liger-kernel into Megatron or if you could provide relevant tutorials.

Thank you once again for your remarkable work!

Alternatives

No response

Additional context

No response