Add some large model troubleshooting steps #31862
Conversation
|
R: @liferoad |
|
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment |
| #### Pickling errors | ||
|
|
||
| When sharing a model across processes with `large_model=True` or using a custom model handler, Beam sends the input and output data across a process boundary. | ||
| To do this, it uses a serialization method known as [pickling](https://docs.python.org/3/library/pickle.html). |
There was a problem hiding this comment.
shall we issue one warning about this pickling error when large_model=True is set? We could link this guide.
There was a problem hiding this comment.
That is a good idea - I'll follow up more on this when I'm back, either in this PR or a separate one
|
I'll add this in a short follow up pr to make sure we're referencing the site correctly in the error (and since this PR is approved/ready to go) |
|
This came up in a conversation with a user who had this issue
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.