Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: partial @streaming blob support for Python (#747)
Adds partial support for the `@streaming` Smithy trait on blobs, depending on a `actions-and-streaming-stdlibs` branch of Dafny that adds an Actions standard library, and only implemented when targeting Python so far. Event streaming is not technically supported yet because more codegen changes are necessary, but this will be a relatively small delta, as the Dafny types involved are already handled. The motivation behind merging this incomplete work is largely to get the test model coverage into CI, so that it doesn't regress as the feature is completed, especially when Dafny verification is perturbed in future versions. The next priority will be to complete the Dafny branch and merge it to Dafny mainline. Note that `smithy-dafny` will still emit a `DANGER` validation event saying that `@streaming` is not supported, which is intentional in order to guard against using the feature in production before it is fully completed: the test models exercising it explicitly suppress this event. * Added a `dafny` submodule under `TestModels/dafny-dependencies/dafny` tracking the `actions-and-streaming-stdlibs` branch, just to access the contents of `Source/DafnyStandardLibraries/src/Std/Actions` etc. This copy of Dafny is never built or used for any other building tasks. * Added a build task under `StandardLibrary` to build a .doo file from a subset of the Dafny standard library files above (using `DafnyStandardLibraries-smithy-dafny-subset.toml`). This is then used as a `--library` argument when verifying/translating polymorph libraries. This approach allows the standard library source to be built with different but compatible options such as `--reads-clauses-on-methods`. * This is guarded by a `USE_DAFNY_STANDARD_LIBRARIES` Makefile variable, since a newer Dafny is required for this to work. * Bumped the high watermark Dafny version tested in CI to 4.9.1, which is necessary since the standard library above uses newer features like `@`-attributes. * Modified Dafny and Python codegen to emit references to the `ByteStream` and `RewindableByteStream` types, and removed the previous workaround to first `.read()` all data in Python. * Because streams have mutable state, the Dafny code generation needs to take them into account when generating things like `modifies` clauses. I haven't yet implemented this, instead using a patch file to add the missing specification to the `Streaming` test model code. * Modified the `aws-sdk/s3` test model and fleshed the `Streaming` test model out to actually test the feature. * Removes not-grep, since our globs are actually completely broken and the tool doesn't support our current code layout, and it was complaining about the contents of the `dafny` submodule. See #763. Reviewer note: it is worth reading the contents of the pending [Actions](https://github.com/dafny-lang/dafny/tree/actions-and-streaming-stdlibs/Source/DafnyStandardLibraries/src/Std/Actions) and [Streams](https://github.com/dafny-lang/dafny/tree/actions-and-streaming-stdlibs/Source/DafnyStandardLibraries/src/Std/Streams) Dafny libraries, but they are still under construction and I'm not claiming they will not change before being merged. Interface feedback is welcome but is not intended to be a hard requirement in scope for this review.
- Loading branch information
