Skip to content

Commit

Permalink
Merge branch 'main' into fix
Browse files Browse the repository at this point in the history
  • Loading branch information
@asherf
asherf authored Jan 9, 2025
2 parents 62d201d + 9a13859 commit 198dda8
Show file tree
Hide file tree
Showing 12 changed files with 1,258 additions and 972 deletions.
56 changes: 29 additions & 27 deletions designs/interruption-handling.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,15 +29,17 @@ There are two ways in-which Spot interruption notifications and Rebalance Recomm
EC2 IMDS is an HTTP API that can only be locally accessed from an EC2 instance.

```
`curl 169.254.169.254/latest/meta-data/spot/instance-action
# Termination Check
curl 169.254.169.254/latest/meta-data/spot/instance-action
{
"action": "terminate",
"time": "2022-07-11T17:11:44Z"
}
curl 169.254.169.254``/``latest``/``meta``-``data``/``events``/``recommendations``/``rebalance`
`{`
` ``"noticeTime"``:`` ``"2022-07-16T19:18:24Z"`
# Rebalance Check
curl 169.254.169.254/latest/meta-data/events/recommendations/rebalance
{
"noticeTime": "2022-07-16T19:18:24Z"
}
```
Expand All @@ -47,19 +49,19 @@ curl 169.254.169.254``/``latest``/``meta``-``data``/``events``/``recommendations
EventBridge is an Event Bus service within AWS that allows users to set rules on events to capture and then target destinations for those events. Relevant targets for Spot interruption notifications include SQS, Lambda, and EC2-Terminate-Instance.

```
`# Example spot interruption notification EventBridge rule`
`$ aws events put``-``rule \`
` ``--``name ``MyK8sSpotTermRule`` \`
` ``--``event``-``pattern ``"{\"source\": [\"aws.ec2\"],\"detail-type\": [\"EC2 Spot Instance Interruption\"]}"`
`# Example rebalance recommendation EventBridge rule``
$ aws events put-rule \
--name MyK8sRebalanceRule \
--event-pattern "{\"source\": [\"aws.ec2\"],\"detail-type\": [\"EC2 Instance Rebalance Recommendation\"]}"
`` `
`# Example targeting an SQS queue`
`$ aws events put``-``targets ``--``rule ``MyK8sSpotTermRule`` \`
` ``--``targets ``"Id"``=``"1"``,``"Arn"``=``"arn:aws:sqs:us-east-1:123456789012:MyK8sTermQueue"`` `
# Example spot interruption notification EventBridge rule
aws events put-rule \
--name MyK8sSpotTermRule \
--event-pattern "{\"source\": [\"aws.ec2\"],\"detail-type\": [\"EC2 Spot Instance Interruption\"]}"
# Example rebalance recommendation EventBridge rule
aws events put-rule \
--name MyK8sRebalanceRule \
--event-pattern "{\"source\": [\"aws.ec2\"],\"detail-type\": [\"EC2 Instance Rebalance Recommendation\"]}"
# Example targeting an SQS queue
aws events put-targets --rule MyK8sSpotTermRule \
--targets "Id=1,Arn=arn:aws:sqs:us-east-1:123456789012:MyK8sTermQueue"
```


Expand Down Expand Up @@ -113,17 +115,17 @@ SQS exposes a VPC Endpoint which will fulfill the isolated VPC use-case.
Dynamically creating the SQS infrastructure and EventBridge rules means that Karpenter’s IAM role would need permissions to SQS and EventBridge:

```
`"sqs:GetQueueUrl",`
`"sqs:ListQueues"``,`
`"sqs:ReceiveMessage"``,`
`"sqs:CreateQueue"``,`
`"sqs:DeleteMessage"``,`
`"events:ListRules",`
"`events:DescribeRule`",
"events:PutRule",
"sqs:GetQueueUrl",
"sqs:ListQueues",
"sqs:ReceiveMessage",
"sqs:CreateQueue",
"sqs:DeleteMessage",
"events:ListRules",
"events:DescribeRule",
"events:PutRule",
"events:PutTargets",
"`events:DeleteRule`",
`"events:RemoveTargets"`
"events:DeleteRule",
"events:RemoveTargets"
```

The policy can be setup with a predefined name based on the cluster name. For example, `karpenter-events-${CLUSTER_NAME}` which would allow for a more constrained resource policy.
Expand Down
60 changes: 30 additions & 30 deletions go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,20 @@ go 1.23.2

require (
github.com/Pallinder/go-randomdata v1.2.0
github.com/PuerkitoBio/goquery v1.10.0
github.com/PuerkitoBio/goquery v1.10.1
github.com/avast/retry-go v3.0.0+incompatible
github.com/aws/aws-sdk-go-v2 v1.32.6
github.com/aws/aws-sdk-go-v2/config v1.28.6
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.21
github.com/aws/aws-sdk-go-v2/service/ec2 v1.197.0
github.com/aws/aws-sdk-go-v2/service/eks v1.54.0
github.com/aws/aws-sdk-go-v2/service/fis v1.31.2
github.com/aws/aws-sdk-go-v2/service/iam v1.38.2
github.com/aws/aws-sdk-go-v2/service/pricing v1.32.7
github.com/aws/aws-sdk-go-v2/service/sqs v1.37.2
github.com/aws/aws-sdk-go-v2/service/ssm v1.56.1
github.com/aws/aws-sdk-go-v2/service/sts v1.33.2
github.com/aws/aws-sdk-go-v2/service/timestreamwrite v1.29.8
github.com/aws/aws-sdk-go-v2 v1.32.7
github.com/aws/aws-sdk-go-v2/config v1.28.7
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.22
github.com/aws/aws-sdk-go-v2/service/ec2 v1.198.1
github.com/aws/aws-sdk-go-v2/service/eks v1.56.0
github.com/aws/aws-sdk-go-v2/service/fis v1.31.3
github.com/aws/aws-sdk-go-v2/service/iam v1.38.3
github.com/aws/aws-sdk-go-v2/service/pricing v1.32.8
github.com/aws/aws-sdk-go-v2/service/sqs v1.37.4
github.com/aws/aws-sdk-go-v2/service/ssm v1.56.2
github.com/aws/aws-sdk-go-v2/service/sts v1.33.3
github.com/aws/aws-sdk-go-v2/service/timestreamwrite v1.29.9
github.com/aws/karpenter-provider-aws/tools/kompat v0.0.0-20240410220356-6b868db24881
github.com/aws/smithy-go v1.22.1
github.com/awslabs/amazon-eks-ami/nodeadm v0.0.0-20240229193347-cfab22a10647
Expand All @@ -26,8 +26,8 @@ require (
github.com/imdario/mergo v0.3.16
github.com/jonathan-innis/aws-sdk-go-prometheus v0.1.1
github.com/mitchellh/hashstructure/v2 v2.0.2
github.com/onsi/ginkgo/v2 v2.22.0
github.com/onsi/gomega v1.36.1
github.com/onsi/ginkgo/v2 v2.22.2
github.com/onsi/gomega v1.36.2
github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/pelletier/go-toml/v2 v2.2.3
github.com/prometheus/client_golang v1.20.5
Expand All @@ -49,16 +49,16 @@ require (

require (
github.com/Masterminds/semver/v3 v3.2.1 // indirect
github.com/andybalholm/cascadia v1.3.2 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.17.47 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.25 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.25 // indirect
github.com/andybalholm/cascadia v1.3.3 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.17.48 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.26 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.26 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.1 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/endpoint-discovery v1.10.6 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.6 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.24.7 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.6 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/endpoint-discovery v1.10.7 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.7 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.24.8 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.7 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
Expand All @@ -76,7 +76,7 @@ require (
github.com/google/gnostic-models v0.6.8 // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/pprof v0.0.0-20241029153458-d1b30febd7db // indirect
github.com/google/pprof v0.0.0-20241210010833-40e02aabc2ad // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
Expand All @@ -99,15 +99,15 @@ require (
github.com/spf13/cobra v1.8.1 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/x448/float16 v0.8.4 // indirect
golang.org/x/net v0.30.0 // indirect
golang.org/x/net v0.33.0 // indirect
golang.org/x/oauth2 v0.23.0 // indirect
golang.org/x/sys v0.26.0 // indirect
golang.org/x/term v0.25.0 // indirect
golang.org/x/text v0.20.0 // indirect
golang.org/x/sys v0.28.0 // indirect
golang.org/x/term v0.27.0 // indirect
golang.org/x/text v0.21.0 // indirect
golang.org/x/time v0.8.0 // indirect
golang.org/x/tools v0.26.0 // indirect
golang.org/x/tools v0.28.0 // indirect
gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect
google.golang.org/protobuf v1.35.1 // indirect
google.golang.org/protobuf v1.36.1 // indirect
gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
Expand Down
Loading

0 comments on commit 198dda8

Please sign in to comment.