Fix ernie ci auto trainer error #9758
Conversation
|
Thanks for your contribution! |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #9758 +/- ##
===========================================
- Coverage 52.70% 52.37% -0.33%
===========================================
Files 731 727 -4
Lines 117313 115146 -2167
===========================================
- Hits 61827 60306 -1521
+ Misses 55486 54840 -646 ☔ View full report in Codecov by Sentry. |
paddlenlp/trainer/auto_trainer.py
Outdated
|
|
||
| auto_dist_degree = { | ||
| "tensor_parallel": training_args.tensor_parallel_degree > 1, | ||
| "sequence_parallel": sequence_parallel, |
There was a problem hiding this comment.
already move sequence_parallel in training_args, so use "sequence_parallel": training_args.sequence_parallel, directly, no need line 110 above?
| ) | ||
| # NOTE(zhangwl):in pipeline mode , param my be initialized before while delte init_func ,but param is still not is_initialized | ||
| if not param._is_initialized() and param._init_func is not None: | ||
| param.initialize() |
There was a problem hiding this comment.
if param._init_func is not None, should use param._init_func() or model. _init_weights(Layer) ?
| if ( | ||
| kwargs.get("args", None) is not None | ||
| and kwargs["args"].use_intermediate_api | ||
| and not parallelize.has_parallelized_model |
There was a problem hiding this comment.
put judgment not parallelize.has_parallelized_model into this branch, the judgment here is to determine whether to use the basic API or the intermediate API
paddlenlp/trainer/auto_trainer.py
Outdated
| and kwargs["args"].use_intermediate_api | ||
| and not parallelize.has_parallelized_model | ||
| ): | ||
| if self.auto_dist_config is not None: |
There was a problem hiding this comment.
We can determine parallelize.has_parallelized_model here, if yes, must have auto_dist_config
PR types
Bug fixes
PR changes
Others
Description
修复Ernie ci error bug
允许用户自己控制param_init和使用中层api的时机。