You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When app is open, a model is always loaded. Hence GPU/CPU is always in use. If user is using a large model, it would contribute to low performance of computer and memory leak. Need a way to eject the model when not using the playground.
To Reproduce
Steps to reproduce the behavior:
Download Phi3 medium
Try run generation sample
Have task manager open to monitor GPU usage
16GB is barely enough and no way to eject the model to stop GPU usage
Describe the bug
When app is open, a model is always loaded. Hence GPU/CPU is always in use. If user is using a large model, it would contribute to low performance of computer and memory leak. Need a way to eject the model when not using the playground.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A button to eject the model to free processors.
Originally created by @beth-panx
The text was updated successfully, but these errors were encountered: