Skip to content

Commit

Permalink
Create transient Artifacts.toml to manage gc() better.
Browse files Browse the repository at this point in the history
Now that the `julia` buildkite plugin is calling `gc()` quite
aggressively, we need our rootfs images to look like actual, anchored
artifacts, otherwise they are cleaned up too aggressively.

By creating a (transient) `Artifacts.toml` file, we can better control
the generational nature of the Pkg GC.  If we set our collect delay to,
for instance, 2 days, and each time we use a rootfs we call
`bind_artifact!()` with the rootfs treehash, then we get the desirable
behavior that a rootfs is only deleted if it has not been used in the
last 2 days.
  • Loading branch information
staticfloat committed Sep 30, 2021
1 parent c88b24c commit 15d7f0e
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions lib/generate_sandboxed_bash.jl
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,14 @@ if !Pkg.Artifacts.artifact_exists(rootfs_treehash)
Pkg.Artifacts.download_artifact(rootfs_treehash, rootfs_url, nothing; verbose);
end

# Create an Artifacts.toml file pointing to this rootfs treehash.
# The file itself will be deleted after this pipeline finishes, but
# depending on the `collect_delay` set in the pipeline, it may stay
# around for a while. This works because `bind_artifact!()` internally
# writes into `artifact_usage.toml` in our depot, which informs `gc()`.
temp_artifact_toml = joinpath(mktempdir(prefix="sandbox-buildkite-plugin", cleanup=false), "Artifacts.toml")
Pkg.Artifacts.bind_artifact!(temp_artifact_toml, "rootfs", rootfs_treehash)

# Helper to map in a directory that is defined within an environment variable
# if that variable is set, and it's pointing to a real directory.
function add_env_dir!(dict, env_var; default=nothing)
Expand Down

0 comments on commit 15d7f0e

Please sign in to comment.