You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hello, I have encountered training node collapse and I want to resume the training, I am wondering if megatron would reusme the datasets (i.e., not see the tokens have been trained) and if so how does it work and is there any argument in arguments.py related to that? Very appreciate it if someone can help~
The text was updated successfully, but these errors were encountered:
Hello, I have encountered training node collapse and I want to resume the training, I am wondering if megatron would reusme the datasets (i.e., not see the tokens have been trained) and if so how does it work and is there any argument in arguments.py related to that? Very appreciate it if someone can help~
The text was updated successfully, but these errors were encountered: