❗ CDK CLI versions and CDK library versions will soon diverge

[rix0rrr]

Hello everyone,

This issue is an announcement that in a couple of weeks the CDK CLI and the CDK Construct Library will no longer be released in lockstep. Instead, they will both have their own release cadence, which means their version numbers are going to diverge.

This will not fundamentally change the way CDK works or how you use CDK: the newest version of the CLI will always support all versions of the CDK construct library that were released before it. You can continue to freely upgrade the CLI to the latest version whenever you want.

The first new version of the CDK CLI on the new version line will be released as 2.1000.0, the next one as 2.1001.0, etc. The CDK Construct Library will continue its current version line, releasing as 2.174.0, 2.175.0, 2.176.0, etc.

What are we changing?

The big change is that we are moving the source code of the CLI and related components to a new GitHub repository(*).

Fundamentally, the CDK CLI and CDK Construct Library were always separate parts of the development tool we call "CDK". They were historically located in the same repository because that was easy and quick for us to iterate. Now that CDK has matured and is becoming clear that changes to the different components proceed at different paces and require different testing strategies, we think it's best for the continued evolution to make the split more clear and official.

Because there will be two different repositories that we develop in and release from, the CLI and Construct Library will no longer release in lockstep. This means their version numbers will start to drift apart. We don't know yet what the future will bring: whether the CLI will be released more often than the Construct Library, or the other way around, or whether they will both continue to release at roughly the same cadence, will depend on future circumstances. We at least will have the ability to make changes to the release cadence of one subproject without affecting the other one, which will give the whole project more agility.

The fundamental compatibility model of the CLI remains the same: a CLI can process the output of all Construct Libraries released with it or before it. This used to manifest as the rule that CLI version >= Lib version is always a valid combination of versions. Because the versions will no longer be released in lockstep, the new rule will be CLI release date >= Lib release date. It will be not as easy to spot that information at a glance from the version number, but we will include the minimum required CLI version number in the cdk.out manifest itself so that the error message will tell you what version is required, and we will publish a table with the compatibility information of every version to GitHub.

To indicate a break in the version line continuity, we will leave a large and obvious break in the version numbers of the CLI: we will go from 2.174.0 (same version as the construct library) all the way to 2.1000.0, and then proceed with 2.1001.0 etc. This hopefully makes it obvious that the version lines are not related to each other anymore, without us changing the major version number.

In the mean time, CDK Construct Library (arguably the most important component) will continue its current version line, releasing as 2.175.0, 2.176.0, etc.

(*) We will share the name when the migration is done and is ready to be opened to the public.

What are we not changing?

We are not changing the major version number: we are not releasing CDK CLI 3.x to signify this break in versioning continuity, for two reasons:

• Most customers will have a dependency range like "aws-cdk": "^2.174.0" in their projects. If we changed the major version number to 3.x, those projects would stop automatically consuming CLI updates and on the next schema change everyone would get a CLI compatibility error. By remaining in major version 2, new releases will still match the specified dependency range and be automatically installed.
• Even though doing so would not in any way signify a "CDKv3" release, we are worried that there is too much room for confusion if we were to label this user-facing component with a major version of 3.

Both of these reasons are why we are opting to put a conspicuous break in the minor version number instead.

Where you go to file issues: even though the CDK CLI code will move to a different repository, and Pull Requests will have to be filed against that other repository, as a user you will continue to report your issues to the current aws/aws-cdk repository. If you, as a customer, are experiencing problems with CDK-the-product, you will continue to come to the same place to report it: the repository of the most important component: the construct library. It will be our job to figure out what component your issue is caused by, and we will create issues in other repositories if and when necessary. This is the same operating procedure as the one we employ for CDK issues caused by jsii.

The compatibillity model: the compatibility model does not change. Future versions of the CLI will always be able to read all cdk.out directories that are being produced by the Construct Library of today. To be assured of compatibility, we recommend that you npm upgrade your CLI version at least as often as you upgrade your Construct Library version: upgrade both at the same time, or upgrade the CLI version more often.

A simple rule to follow would be that CLI release date >= Lib release date is guaranteed to work. A slightly more correct but harder to express rule is that the most recent CLI release before the lib release is guaranteed to work, and anything newer as well.

What is the impact to you?

As a user: you will notice that the versions of the CDK CLI and the CDK Construct Library are no longer the same. We recommend that you think of the version of the CDK Construct Library as "the" version of CDK, because that is the version that will impact your day-to-day experience with CDK the most. We also recommend that you keep your CDK CLI version at the latest version to make sure you always have a CLI version that supports the construct library version you are using. Any scripts that assume that there is a single "CDK version" that you can install both the construct library and the CLI under, will need to be rewritten:

# This no longer works, there will be two different versions
$ CDK_VERSION=2.714.0
$ npm install aws-cdk-lib@$CDK_VERSION
$ npm install aws-cdk@$CDK_VERSION

# Do this instead (install the latest 2.x)
$ npm install aws-cdk@^2

As a contributor: CLI-related issues should continue to be filed in the aws-cdk repository, but you will need to make any Pull Requests to the new repository. Changes that involve both the CLI and the construct library will need to be sent to both repositories, and merged separately. The CLI change will need to be released before the construct library PR can be merged. Further details on the exact workflow will follow in the new repo.

[byF]

This should be a major CDK CLI release. This "divergence" will be confusing either way.

[rix0rrr]

This should be a major CDK CLI release

I hear you, but I think the practical argument that we would introduce upgrade problems for the 99% of projects that will have "aws-cdk": "^2" in their package.json is the stronger one and one we can't ignore. I've changed the order of the motivation around to reflect that.

From your response I can tell that package.json updates and reasoning about SemVer and staying on top of dependency changes doesn't faze you 🙂 but we also have to cater to the customer base for whom this doesn't all come so naturally, and think about how to minimize the impact for them.

[gshpychka]

Good change - hopefully this will alleviate some of the confusion I often see with people not getting the difference between aws-cdk and aws-cdk-lib right away.

As far as I can tell, most releases of aws-cdk contain no changes and are only released to match the aws-cdk-lib versioning and release cadence.

[aripalo]

Very good. As the aws/aws-cdk repo is such a beast and painful to understand & contribute to… so hopefully after separated repos (for the lib and the CLI), it makes contributing a tiny bit easier at least 🤞

[aripalo]

I'm wondering, is recommending aws-cdk@latest a good idea? Would aws-cdk@2 be better recommendation? I.e. requiring to keep the major versions "in sync"?

I'm just thinking of a "far future" situation, in case there ever is a aws-cdk-lib@3 and aws-cdk@3 (which would then became latest).

If going with the recommendation of aws-cdk@latest, then that implies if (I know, a really huuuge if) there's ever v3 major release, then aws-cdk@3 should also be backwards compatible with aws-cdk-lib@2…

[mrpackethead]

This should be a major CDK CLI release

I hear you, but I think the practical argument that we would introduce upgrade problems for the 99% of projects that will have "aws-cdk": "^2" in their package.json is the stronger one and one we can't ignore. I've changed the order of the motivation around to reflect that.

From your response I can tell that package.json updates and reasoning about SemVer and staying on top of dependency changes doesn't faze you 🙂 but we also have to cater to the customer base for whom this doesn't all come so naturally, and think about how to minimize the impact for them.

The 'purest' in me, doesn't like this, but I the practical part of me, see this saving many 'it does'tn work' calls/messages that changing to v3 would cause.

Thanks for another clear update about the future of cdk rico. Timely, ( probably only a few hours ), simple and without the clutter of a monthly meeting.

[dreamorosi]

I'd like to ask for clarification on this point:

The fundamental compatibility model of the CLI remains the same: a CLI can process the output of all Construct Libraries released with it or before it.

Specifically in relation to the fact that, as mentioned, releases of aws-cdk might be slower than aws-cdk-lib.

Does the statement above means that if we update aws-cdk-lib but there hasn't been any release for the CLI, we won't be able to use it?

[rix0rrr]

Does the statement above means that if we update aws-cdk-lib but there hasn't been any release for the CLI, we won't be able to use it?

Thank you for asking. No, that would be awful and that will never be the case. If there is a necessity for the CLI to be updated to support new library features, then of course we will make sure that this release happens!

If going with the recommendation of aws-cdk@latest, then that implies if (I know, a really huuuge if) there's ever v3 major release, then aws-cdk@3 should also be backwards compatible with aws-cdk-lib@2…

Yes, and I think this is what should happen. In fact, when we were in the upgrade from v1 to v2, the v2 CDK CLI also deployed v1 apps. The construct library and framework really make the "CDK version", the CLI is just an attendant tool that happens to "speak CDK". I want users to be able to update their CLI fearlessly without worry about compatibility.

[dreamorosi]

Thank you for clarifying.

Follow up question: what will happen to the @aws-cdk/cli-lib-alpha package?

Will it stay under aws-cdk-lib, go in the new CLI repo, or live in a third separate repo? And what about its versioning strategy?

[rix0rrr]

The @aws-cdk/cli-lib-alpha package is an alternative way of using the CLI, and so belongs with the CLI. It will move with the CLI to the new repo.

In fact we are working on a refactor of this package right now.

I'm not 100% positive on its versioning strategy off the top of my head; I think it makes sense if it would mimic the CLI's version exactly, but I'll let @mrgrain have the last word on that.

[mrgrain]

In fact we are working on a refactor of this package right now.

I'm not 100% positive on its versioning strategy off the top of my head; I think it makes sense if it would mimic the CLI's version exactly, but I'll let @mrgrain have the last word on that.

I'm not convinced yet the programmatic toolkit and CLI should use the same version as they are exposing similar but different APIs. Locking them together creates a similar problem we had with CLI and aws-cdk-lib albeit less prominent. Currently I see the CLI as a very stable wrapper around the Toolkit that is intended for advanced users and we place where can assume familiarity with dependency management and semantic versioning choices.

Either way, in the short term, the Toolkit should use a 0.x version until we have broad support of features and API maturity.

[rix0rrr]

After discussing with @mrgrain, I'll adjust my position. You are both right, we should indeed recommend:

$ npm install aws-cdk@^2

And I've updated the examples to match.

• Not because CLI v3 would not be compatible with LIB v2. It should be and will be!
• But because CLI v3 might have a different UX than CLI v2, and we don't want people to consume the new CLI with potentially different UX in an unexpected fashion.