Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PFI (Permutation Feature Importance) takes forever on some regression models created using AutoML #7345

Open
lolsoftware opened this issue Dec 17, 2024 · 0 comments
Labels
untriaged New issue has not been triaged

Comments

@lolsoftware
Copy link

System Information

  • Windows 11
  • ML.NET v4.0.0
  • .NET 8.0

Describe the bug
Calculating PFI seems to hang (take forever) for some regression models created using AutoML. For other models (created against the same learning data) calculating PFI takes just a couple of seconds.

To Reproduce
Run the attached program.

Expected behavior
I would expect PFI to complete in a timely fashion. Or at least provide a mechanism to monitor the progress of PFI calculation with the ability to cancel it.

Screenshots, Code, Sample Projects
The attached program contains two regression models created using AutoML, and a set of data used to create the two models. The program first calculates PFI against the first model which takes just a couple of seconds. Then the program tries to calculate PFI against the second model which takes forever.
test project.zip

@dotnet-policy-service dotnet-policy-service bot added the untriaged New issue has not been triaged label Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
untriaged New issue has not been triaged
Projects
None yet
Development

No branches or pull requests

1 participant