Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: reproducible choice interactive #262

Merged
merged 32 commits into from
Dec 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
0794bf3
remove details; untie deploy from BuildCommand; init prototyping
Nov 14, 2024
9bae407
feat: structure of cargo near `build` command with interactive subcom…
Nov 14, 2024
98d7f47
feat: structure of `deploy` command with interactive subcommand
Nov 18, 2024
3b59734
todo: add todo about flipping `no-locked` sign in `cargo_near_build::…
Nov 19, 2024
301f9b8
feat: remove redundant fields of opts in lib, related to `passthrough…
Nov 20, 2024
5feda26
chore: remove mentions of `--no-docker` flag (doc, tests pending)
Nov 20, 2024
8c95855
chore: `Opts.cli_description` - flip `locked`, change default prefix
Nov 20, 2024
b304f3d
feat: join back build cmd + lib build opts
Nov 20, 2024
eb47fa1
chore: text of interactive choice and --help
Nov 21, 2024
6e44f83
feat: enforce `cargo near build non-reproducible-wasm` from within do…
Nov 21, 2024
0f3f053
chore: rename `build_command` module to `build`
Dec 3, 2024
32a8d72
chore: rename `abi_command` module to `abi`
Dec 3, 2024
2ffba00
chore: link `deploy` command back to `build` actions
Dec 3, 2024
b0e23aa
chore: edit `--help` and interactive help msgs
Dec 3, 2024
29a114d
feat: resolve TODO with enforcing corrent command call in docker
Dec 3, 2024
059ca93
chore: prettier indents in interactive help msgs
Dec 4, 2024
3a2276d
chore: improve message of parsing docker specific metadata
Dec 4, 2024
499ccc1
chore: flip `--no-locked` flag for `abi` command
Dec 4, 2024
333a916
test: fix tests as per changes of api and logic (interactive build ch…
dj8yfo Dec 5, 2024
c592c1b
chore: highlight custom metadata section in another msg
Dec 9, 2024
4988f5f
chore: change `.github/workflows` of `cargo near new` project (intera…
dj8yfo Dec 9, 2024
2732825
doc: edit README.md for reproducible choice interactive change (#265)
dj8yfo Dec 11, 2024
2dd3c29
chore: rename `AbiCommand` -> `Command` as well
Dec 11, 2024
cba637c
chore: fix reference to structs' names in comments
Dec 11, 2024
a474b0d
bump: bump `near-cli-rs` with `--teach-me` improvement
Dec 12, 2024
9682307
review: addresses https://github.com/near/cargo-near/pull/262/files#r…
Dec 16, 2024
ead640e
review: addresses https://github.com/near/cargo-near/pull/262/files#r…
Dec 16, 2024
3383684
review: addresses https://github.com/near/cargo-near/pull/262/files#r…
Dec 16, 2024
ecaa41e
review: addresses https://github.com/near/cargo-near/pull/262/files#r…
Dec 16, 2024
6a0013d
review: addresses https://github.com/near/cargo-near/pull/262/files#r…
Dec 16, 2024
615fefe
doc: edit text about local/production in README
Dec 16, 2024
ad6c8a4
ci: fix doc job
Dec 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/crontab_new_template_renewal.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ jobs:
let record = http get "https://hub.docker.com/v2/namespaces/sourcescan/repositories/cargo-near/tags" | get results | first;

let mod_content = (
open cargo-near/src/commands/new/new-project-template/Cargo.toml.template --raw | lines
open cargo-near/src/commands/new/new-project-template/Cargo.template.toml --raw | lines
| each {
|line| if ($line | str starts-with "image = ") {
$'image = "sourcescan/cargo-near:($record.name)"'
Expand All @@ -44,7 +44,7 @@ jobs:
| to text
);

$mod_content | save -f cargo-near/src/commands/new/new-project-template/Cargo.toml.template
$mod_content | save -f cargo-near/src/commands/new/new-project-template/Cargo.template.toml

git diff

Expand Down
5 changes: 2 additions & 3 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

75 changes: 53 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,26 +99,57 @@ cargo near build

Builds a NEAR smart contract along with its [ABI](https://github.com/near/abi) (while in the directory containing contract's Cargo.toml).

By default, this runs a reproducible build in a [Docker](https://docs.docker.com/) container, which:
Running the above command opens a menu with following variants:

### `non-reproducible-wasm`

**Recommended variant for use during local development.**

This is a regular build, which behaves much like and is a thin wrapper around a regular `cargo build --target wasm32-unknown-unknown --release`.

Additional flags for build configuration can be looked up by

```bash
cargo near build non-reproducible-wasm --help
```
if needed.

### `reproducible-wasm`

**Recommended variant for the production releases.**

This variant runs a reproducible build in a [Docker](https://docs.docker.com/) container, which:

1. runs against source code version, committed to git, ignoring any uncommitted changes
2. requires that `Cargo.lock` of project is created (e.g. via `cargo update`) and added to git.
- this enables `--locked` build by downstream `cargo` command.
3. will use configuration in [`[package.metadata.near.reproducible_build]`](https://github.com/near/cargo-near/blob/main/cargo-near/src/commands/new/new-project-template/Cargo.toml.template#L14-L25)
section of contract's `Cargo.toml` and [`package.repository`](https://github.com/near/cargo-near/blob/main/cargo-near/src/commands/new/new-project-template/Cargo.toml.template#L9) field
3. will use configuration in [`[package.metadata.near.reproducible_build]`](https://github.com/near/cargo-near/blob/main/cargo-near/src/commands/new/new-project-template/Cargo.template.toml#L14-L25)
section of contract's `Cargo.toml` and [`package.repository`](https://github.com/near/cargo-near/blob/main/cargo-near/src/commands/new/new-project-template/Cargo.template.toml#L9) field
- default values for this section can also be found in `Cargo.toml` of
template project, generated by `cargo near new`

Important flags:
**What's a reproducible build in context of NEAR?**
Why is it needed? Explanation of these points and a step-by-step tutorial is present at [SourceScan/verification-guide](https://github.com/SourceScan/verification-guide).

1. `--no-docker`
- flag can be used to perform a regular build with rust toolchain installed onto host, running the `cargo-near` cli.
- *NO*-Docker builds run against actual state of code in filesystem and not against a version, committed to source control.

2. `--no-locked`
- flag is allowed in *NO*-Docker builds, e.g. to generate a `Cargo.lock` *and* simultaneously build the contract.
- flag is allowed in Docker builds, but
- such builds are not reproducible due to potential update of dependencies and compiled `wasm` mismatch as the result.
<details>
<summary>Additional (optional) details on possible <code>[package.metadata.near.reproducible_build]</code> configuration</summary><p>

1. available images can be found by this link https://hub.docker.com/r/sourcescan/cargo-near/tags
- [`image`](https://github.com/near/cargo-near/blob/main/cargo-near/src/commands/new/new-project-template/Cargo.template.toml#L18) and [`image_digest`](https://github.com/near/cargo-near/blob/main/cargo-near/src/commands/new/new-project-template/Cargo.template.toml#L19) are straightforward to configure:
![image_and_digest_pinpoint](./docs/image_and_digest_pinpoint.png)
2. flags of build command, run inside of docker container, can be configured, if needed, by changing [`container_build_command`](https://github.com/near/cargo-near/blob/main/cargo-near/src/commands/new/new-project-template/Cargo.template.toml#L29) field
- base `container_build_command` for images starting with **sourcescan/cargo-near:0.13.0-rust-1.83.0** and after it is `["cargo", "near", "build", "non-reproducible-wasm", "--locked"]`, where the `--locked` flag is required
- base `container_build_command` for images prior to **sourcescan/cargo-near:0.13.0-rust-1.83.0** is `["cargo", "near", "build"]`
- additional flags, if needed, can be looked up on
- `cargo near build non-reproducible-wasm --help` for newer/latest images
- `cargo near build --help` for older ones
- running `docker run -it sourcescan/cargo-near:0.11.0-rust-1.82.0` (or another specific image) and checking the `--help` message of exact `cargo-near` in container may be helpful when in doubt
3. `cargo near` allows parameterizing build with values of environment variables, present at the time of the build and not present in a contract's source code,
by specifying their names in [`passed_env`](https://github.com/near/cargo-near/blob/main/cargo-near/src/commands/new/new-project-template/Cargo.template.toml#L24) array
- supported by **sourcescan/cargo-near:0.10.1-rust-1.82.0** image or later images
- SourceScan/Nearblocks does not support verifying such contracts with additional parameters present in their metadata yet

</p></details>

---

Expand Down Expand Up @@ -146,22 +177,22 @@ cargo near deploy

Builds the smart contract (equivalent to `cargo near build`) and guides you to deploy it to the blockchain.

By default, this runs a reproducible build in a Docker container.
Similar to `build`, running the above command opens a menu with following variants:

### `build-non-reproducible-wasm`

This forwards to [non-reproducible-wasm](#non-reproducible-wasm) variant of `build` command.

### `build-reproducible-wasm`

This forwards to [reproducible-wasm](#reproducible-wasm) variant of `build` command.

`deploy` command from Docker build requires that contract's source code:

1. doesn't have any modified tracked files, any staged changes or any untracked content.
2. has been pushed to remote repository, identified by
[`package.repository`](https://github.com/near/cargo-near/blob/main/cargo-near/src/commands/new/new-project-template/Cargo.toml.template#L9).

Important flags:

1. `--no-docker`
- flag can be used to perform a regular *NO*-Docker build *and* deploy.
- Similar to `build` command, in this case none of the git-related concerns and restrictions apply.
[`package.repository`](https://github.com/near/cargo-near/blob/main/cargo-near/src/commands/new/new-project-template/Cargo.template.toml#L9).

2. `--no-locked`
- flag is declined for deploy, due to its effects on `build` result

## Contribution

Expand Down
11 changes: 0 additions & 11 deletions cargo-near-build/src/near/docker_build/docker_checks/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ pub fn handle_command_io_error<T>(
println!();
println!("{}", "`docker` executable isn't available".yellow());
print_installation_links();
print_non_docker_suggestion();
Err(report)
}
Err(io_err) => {
Expand All @@ -33,7 +32,6 @@ pub fn handle_command_io_error<T>(
.yellow()
);
println!("{}", format!("Error `{:?}`", io_err).yellow());
print_non_docker_suggestion();
Err(report)
}
}
Expand Down Expand Up @@ -127,13 +125,4 @@ pub fn print_command_status(status: std::process::ExitStatus, command: std::proc
)
.yellow()
);
print_non_docker_suggestion();
}

fn print_non_docker_suggestion() {
println!(
"{}",
"You can choose to opt out into non-docker build behaviour by using `--no-docker` flag."
.cyan()
);
}
13 changes: 9 additions & 4 deletions cargo-near-build/src/near/docker_build/mod.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
use std::process::{Command, ExitStatus};

use colored::Colorize;

use crate::docker::DockerBuildOpts;
use crate::types::near::build::input::BuildContext;
use crate::types::near::build::output::CompilationArtifact;
Expand Down Expand Up @@ -33,10 +35,13 @@ pub fn run(opts: DockerBuildOpts) -> eyre::Result<CompilationArtifact> {
},
)?;

let docker_build_meta =
pretty_print::handle_step("Parsing and validating `Cargo.toml` metadata...", || {
metadata::ReproducibleBuild::parse(cloned_repo.crate_metadata())
})?;
let docker_build_meta = pretty_print::handle_step(
&format!(
"Parsing and validating `{}` section of contract's `Cargo.toml` ...",
"[package.metadata.near.reproducible_build]".magenta()
),
|| metadata::ReproducibleBuild::parse(cloned_repo.crate_metadata()),
)?;

if let BuildContext::Deploy {
skip_git_remote_check,
Expand Down
4 changes: 3 additions & 1 deletion cargo-near-build/src/types/cargo/metadata.rs
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,9 @@ fn get_cargo_metadata(
);
println!(
"{}",
"You can choose to disable `--locked` flag for downstream `cargo` command with `--no-locked` flag.".cyan()
"You can choose to disable `--locked` flag for downstream `cargo` command \
by adding `--no-locked` flag OR by removing `--locked` flag"
.cyan()
);
thread::sleep(Duration::new(5, 0));
return Err(cargo_metadata::Error::CargoMetadata {
Expand Down
20 changes: 15 additions & 5 deletions cargo-near-build/src/types/near/build/input/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,12 @@ impl Default for CliDescription {
fn default() -> Self {
Self {
cli_name_abi: "cargo-near".into(),
cli_command_prefix: vec!["cargo".into(), "near".into(), "build".into()],
cli_command_prefix: vec![
"cargo".into(),
"near".into(),
"build".into(),
"non-reproducible-wasm".into(),
],
}
}
}
Expand All @@ -97,13 +102,16 @@ impl Opts {
/// this is just 1-to-1 mapping of each struct's field to a cli flag
/// in order of fields, as specified in struct's definition.
/// `Default` implementation corresponds to plain `cargo near build` command without any args
pub(crate) fn get_cli_command_for_lib_context(&self) -> Vec<String> {
pub fn get_cli_command_for_lib_context(&self) -> Vec<String> {
let cargo_args = self.cli_description.cli_command_prefix.clone();
let mut cargo_args: Vec<&str> = cargo_args.iter().map(|ele| ele.as_str()).collect();
if self.no_locked {
cargo_args.push("--no-locked");
// this logical NOT is needed to avoid writing manually `Default` trait impl for `Opts`
// with `self.locked` field and to keep default (if nothing is specified) to *locked* behavior
// which is a desired default for [crate::extended::build] functionality
if !self.no_locked {
cargo_args.push("--locked");
}
// `no_docker` field isn't present

if self.no_release {
cargo_args.push("--no-release");
}
Expand Down Expand Up @@ -220,6 +228,8 @@ mod tests {
assert_eq!(opts.get_cli_command_for_lib_context(), ["cargo".to_string(),
"near".to_string(),
"build".to_string(),
"non-reproducible-wasm".to_string(),
"--locked".to_string(),
"--env".to_string(),
"KEY=VALUE".to_string(),
"--env".to_string(),
Expand Down
Loading
Loading