Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Raise ValueError for alpha > 1 in sigmoid_focal_loss #8882

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

developer0hye
Copy link
Contributor

Overview

This PR updates the sigmoid_focal_loss function to explicitly raise a ValueError when alpha > 1.
In the focal loss formulation, alpha is meant to balance positive and negative examples and should
generally lie within the range (0, 1) or be set to -1 to ignore the balancing factor.
Allowing alpha to exceed 1 can lead to unexpected behaviors.

Changes

  • Added a condition to check if alpha > 1. If true, it raises a ValueError.
  • Clarified in the docstring that valid values of alpha are within (0, 1) or -1 for ignoring.

Reasoning

  • According to the focal loss paper and its implementations, alpha in the range of (0, 1) (or -1)
    is standard practice.
  • Ensuring alpha does not exceed 1 prevents users from accidentally passing invalid parameters,
    thus avoiding silent errors or unstable training behavior.

Additional Notes

  • If there is a special use case for greater, we can discuss supporting that explicitly.
    However, for most applications, restricting alpha to (0, 1) or -1 remains consistent with the
    original focal loss approach.

- Restrict alpha to (0, 1) or -1 for ignore, as per the original focal loss settings.
- Add a check to raise ValueError when alpha exceeds 1.
- This helps prevent invalid alpha values from silently causing unexpected behaviors.
Copy link

pytorch-bot bot commented Jan 26, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/8882

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants