Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unlocked deps #393

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions design/mvp/WIT.md
Original file line number Diff line number Diff line change
Expand Up @@ -661,6 +661,60 @@ world w2 {
> configure that a `use`'d interface is a particular import or a particular
> export.

## Unlocked Imports (semver)
macovedj marked this conversation as resolved.
Show resolved Hide resolved

When working with a registry, the keyword `unlocked-dep` is available to specify version requirements as ranges.
macovedj marked this conversation as resolved.
Show resolved Hide resolved

```wit
world w {
unlocked-dep foo:bar@{>=x.x.x <y.y.y};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it intended to support foo:bar here exactly? Or is a /interface required as well?

Copy link
Member

@lukewagner lukewagner Sep 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking that, when /interface is absent, the expectation could be that a registry is used to resolve ns:package to a component and the interface is the exports of that component (which have no name).

More hypothetically, if we get unnamed interfaces/worlds in WIT, /interface could also be absent without relying on a registry by using a nested package of the form:

package foo:bar {
  interface { ... }
}
world w {
  unlocked-dep foo:bar@{...}
}

WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But perhaps this is a good opportunity to make clear that the optional /interface is possible as well, and maybe talk a bit about in which cases toolchains would create an unlocked-dep with and without an interface projection? If we wanted to scope it in, folks could specify an interface when they add a dep and tools could only grab interfaces they need from the registry. And also the lock or bundle command could shake out unused interfaces in cases as well. May be more than we want for the initial discussion, but I was thinking that folks may find it confusing to only see a component foo:bar when they're accustomed to depending on interfaces at the moment.

Copy link
Member

@lukewagner lukewagner Sep 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fundamentally, an unlocked-dep (in the component importname) names a component via package name and so, for a simple illustrative example like this, I think makes sense to similarly start with unlocked-dep naming a package. (Yes, it's different that regular interface imports, but being different is the point.) The /interface only shows up after inlining registry contents and is only necessary due to current expressive limitations of WIT; ideally it wouldn't ever be necessary.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this spec out the interface { ... } change as well in that case? That's not currently implemented or sketched out here, so I think that should be included too if that's the intention. (also could this update the ebnf for unlocked-dep in worlds too?)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be up for that; it would simplify things.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great I should be able to add some of these updates soon.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As an alternative to adding unnamed interfaces/worlds (which open up another set of design questions), we could instead add a bit of also-useful syntax that allows the WIT to capture precisely how the range query was resolved:

package gh:sqlite@1.1.1 {
  interface exports { ... }
}
world w {
  import dependency gh:sqlite@{>=1.0.0} = gh:sqlite/[email protected];
}

This = <pkgname> right-hand-side of the import dependency is necessary for advanced cases where the WIT needs to nest multiple versions of the same package such that the <pkgnamequery> alone would be ambiguous. Importantly for our purposes here, though, none of the name/version info in the = <pkgname> shows up in the import so our world w above just contains (import "unlocked-dep=<gh:sqlite@{>=1.0.0}>" (instance ...)) (no /exports, no 1.1.1), and we didn't need anonymous interfaces to achieve that.

}
```

The binary format has a corresponding [import definition](Explainer.md#import-and-export-definitions) and this WIT syntax informs
bindgen tooling that it should be used.

The key idea here is to be able to specify a dependency on a _component_, rather than on a wit interface. Sometimes, as a component author, the goal is to have a dynamic import, where at a time after development is done, one of many implementations of a wit interface is specified, so that in essence your dependency makes your component configurable. This workflow is well documented across a variety of tools. Unlocked imports, on the other hand, are available for
specifying a dependency on a specific implementation of an interface, with a semver range
macovedj marked this conversation as resolved.
Show resolved Hide resolved

### Example Unlocked Workflow
macovedj marked this conversation as resolved.
Show resolved Hide resolved
Each language has its own toolchain for creating wasm components that should feel familiar to users of that language. As an example, somebody authoring a rust component would add the component they're interested in to their `Cargo.toml`.
macovedj marked this conversation as resolved.
Show resolved Hide resolved

```
"foo:bar" = "x.x.x"
macovedj marked this conversation as resolved.
Show resolved Hide resolved
```
macovedj marked this conversation as resolved.
Show resolved Hide resolved

Say that the exports of this component match the exports of some `world exports`. Then the wit used for the component being authored would end up as follows:

```wit
package my:component

package foo:bar {
interface exports {
nest some:other/interfacename
...
}
}

world w {
unlocked-dep foo:bar/exports@{>=x.x.x <y.y.y}
}
```
macovedj marked this conversation as resolved.
Show resolved Hide resolved


Once this wit is synthesized by the language toolchain based on the language's package file, bindings can be generated and a wasm binary can be compiled which will contain unlocked imports as defined in the [import definition section](Explainer.md#import-and-export-definitions) of the explainer. Below is an example of the unlocked imports that will be present in the `wat`

```wat
(import "unlocked-dep=<foo:bar/exports@{>=1.0.0}>")
```
macovedj marked this conversation as resolved.
Show resolved Hide resolved

A wasm component binary that has unlocked imports is referred to as an unlocked component. In general, unlocked components are what would be published to a registry. Registry aware tools have a command `lock` that will produce a "locked" component when it resolves dependency versions. As an example, today the warg cli has a [lock command](https://github.com/bytecodealliance/registry/blob/main/src/commands/lock.rs). This "locked" component will use locked import statements, rather than unlocked import statements, with pinned version numbers and integrity hashes discovered during package resolution, as can be seen below.

```wat
(import "locked-dep=<foo:bar/[email protected]>,integrity=<sha256-7b582e13fd1f798ed86206850112fe01f837fcbf3210ce29eba8eb087e202f62>")
```

Locked components aren't runnable unless they are being run by a registry aware runtime. They serve the role of a reproducible deployment artifact. In order to run them with a runtime that is not registry aware, one would need to use a `bundle` command, also made available by registry aware toolchains, that will inline component definitions where locked imports exist in a locked component. The warg cli also has a reference implementation of a [bundle command](https://github.com/bytecodealliance/registry/blob/main/src/commands/bundle.rs).
macovedj marked this conversation as resolved.
Show resolved Hide resolved
## WIT Functions
[functions]: #wit-functions

Expand Down
Loading