Skip to content

Commit

Permalink
Improve events when max total nodes of the cluster is reached.
Browse files Browse the repository at this point in the history
- log cluster wide event - previous event would never get fired because
  the estimators would already cap the options they generate and
additionally it would fire once and events are kept only for some time
- log per pod event explaining why the scale up is not triggered
  (previously it would either get no scale up because no matching group
or it would not get an event at all)
  • Loading branch information
jbtk committed Jan 7, 2025
1 parent 998e45a commit 15edd9a
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions cluster-autoscaler/core/static_autoscaler.go
Original file line number Diff line number Diff line change
Expand Up @@ -526,6 +526,12 @@ func (a *StaticAutoscaler) RunOnce(currentTime time.Time) caerrors.AutoscalerErr
} else if a.MaxNodesTotal > 0 && len(readyNodes) >= a.MaxNodesTotal {
scaleUpStatus.Result = status.ScaleUpNoOptionsAvailable
klog.V(1).Infof("Max total nodes in cluster reached: %v. Current number of ready nodes: %v", a.MaxNodesTotal, len(readyNodes))
autoscalingContext.LogRecorder.Eventf(apiv1.EventTypeWarning, "MaxNodesTotalReached",
"Max total nodes in cluster reached: %v", autoscalingContext.MaxNodesTotal)
for _, pod := range unschedulablePodsToHelp {
autoscalingContext.Recorder.Event(pod, apiv1.EventTypeNormal, "NotTriggerScaleUp",
fmt.Sprintf("pod didn't trigger scale-up: %s", "max total nodes in cluster reached"))
}
} else if len(a.BypassedSchedulers) == 0 && allPodsAreNew(unschedulablePodsToHelp, currentTime) {
// The assumption here is that these pods have been created very recently and probably there
// is more pods to come. In theory we could check the newest pod time but then if pod were created
Expand Down

0 comments on commit 15edd9a

Please sign in to comment.