IMDS Client Improvements

[commiterate]

Description

The cognitect.aws.ec2-metadata-utils namespace includes a limited IMDS client with a few things like a fixed IMDS session token TTL (set to the max of 6 hours), fixed IMDS IPv4 endpoint, and deprecated get-data and get-data-at-path functions.

IMDS clients in the official AWS SDKs support the options outlined in the IMDS client docs and IMDS session token auto-refresh + caching.

This enhancement is to implement a more feature-complete IMDS client (e.g. in cognitect.aws.client.imds) which can be used for general-purpose IMDS calls.

Usage might look something like this:

(ns my-ns
  (:require
    [cognitect.aws.client.imds :refer [client invoke]]
    [cognitect.aws.client.shared :refer [http-client]])

(def imds
  (client
    {; Default to the values in the IMDS client docs.
     :retries 3
     :port 80
     :token-ttl 21600
     :endpoint-mode :IPv6
     ; Default to the shared HTTP client.
     :http-client http-client}))

;; Low-level operations.

(invoke imds {:op :Get :request {:Path "/latest/meta-data/autoscaling/target-lifecycle-state"}})

;; High-level operations.

; Gets + base64-decodes user data.
(invoke imds {:op :GetUserData})

; Gets region.
(invoke imds {:op :GetRegion})

Use Cases

Using Clojure + GraalVM to write on-instance handlers for EC2 Auto Scaling lifecycle hooks. The target lifecycle state can be retrieved through IMDS on the instance.

These on-instance handlers are run as a daemon (e.g. as a systemd service unit) which may poll IMDS indefinitely.

[marcobiscaro2112]

I think a general-purpose IMDS API is a reasonable request, but I think we should limit such API to a single function that receives the path to issue a request and returns the response body (i.e. no "high-level operations" as you mentioned).

The reason for this is that as far as I know there is no data that could be consumed programmatically to generate (and keep up-to-date) such operations. I think the closest thing to that is this table that loosely document existing operations (but not the exact contracts, like response content type or response body shape).

The Java SDK also does this (only provide a single get method in Ec2MetadataClient).

[commiterate]

That's fair.

I imagine some library internals like the region and credentials provider chain will need their own functions that use hardcoded paths (e.g. /latest/meta-data/placement/region, /latest/meta-data/iam/security-credentials/{role name}) and response deserializers (e.g. assume the IAM role credentials response is JSON). In that case, the hardcoding belongs strictly in those functions or namespaces (e.g. as a let binding or a def ^:private) and shouldn't be accessible by users.

The IMDS API is a weird one because its model isn't done in the same way as AWS services and, unlike other AWS compute environment metadata APIs (e.g. Lambda's runtime + extensions + telemetry APIs), the model isn't distributed publicly.

I have a separate issue on AWS's side to make IMDS modelable in Smithy so it can hook into the regular modeling + SDK generation process AWS services use.

smithy-lang/smithy#2505

Unfortunately, it'll probably be years before the AWS compute environment metadata APIs are all in Smithy and code-generated in the AWS SDKs.