diff --git a/justfile b/justfile index bf5e254e1ab81d1c70dda827475f6aeb836d0121..c7ce2b705cd36d810b148e470d0983574baf30b7 100644 --- a/justfile +++ b/justfile @@ -2,7 +2,7 @@ _default: @just --list # Skills backed up in this repo, installed into ~/.claude/skills (repo -> ~/.claude) -skills := "sourcehut-ci sourcehut-custom-service" +skills := "sourcehut-ci sourcehut-ci-cacher sourcehut-custom-service" # Absolute path to this repo (the SourceHut documentation mirror), substituted # into skills for the __SRHT_MIRROR__ placeholder at install time. diff --git a/skills/sourcehut-ci-cacher/SKILL.md b/skills/sourcehut-ci-cacher/SKILL.md new file mode 100644 index 0000000000000000000000000000000000000000..74b5e9f3a1175110e8c58cbbf2f0fa4b6130b391 --- /dev/null +++ b/skills/sourcehut-ci-cacher/SKILL.md @@ -0,0 +1,362 @@ +--- +name: sourcehut-ci-cacher +description: Add or fix build caching in a builds.sr.ht manifest with either of the user's two cache backends — `cacher` (go.bigb.es/cacher, straight to Garage S3) or `art cache` (the artifacts.sr.ht client, same bucket through the daemon and no S3 credential in the build). Use whenever a .build.yml needs to cache Go module/build trees, an assembled SCSS tree, a docker image or any other directory between builds; whenever a build is slow because it recompiles or re-clones the same dependencies every time; whenever the user mentions cacher, art cache, cache_restore, cache_save, cache_setup, docker-cache, ci-cacher, `~/.s3-cache-key-id`, `~/.srht-token`, or asks how to cache anything on sourcehut; and whenever a manifest is being moved from cacher to art. Also use when copying cache stages between the `~/data/home/sourcehut-*` sibling repos, since those carry two different generations of the pattern and the older one is easy to copy by mistake. Companion to the `sourcehut-ci` skill, which covers builds.sr.ht itself — upstream sourcehut genuinely has no cache feature, and this is the local answer to that. +license: BSD-3-Clause +metadata: + audience: developers + workflow: ci + tags: [ci, sourcehut, builds.sr.ht, cache, cacher, art, artifacts.sr.ht, s3, garage, go] + version: 1.1.0 + author: bigbes +--- + +# Build caching on sourcehut + +builds.sr.ht has no cache feature: every build is a fresh VM. Two local helpers +answer that, and they are interchangeable because they address the same objects: + +- **`cacher`** (`go.bigb.es/cacher`) tars a directory to S3 under a + content-derived key, speaking S3 itself. Needs the Garage key pair. +- **`art cache`** (the artifacts.sr.ht client) does the same through + `artifacts.srht.bigb.es/api/v1/cache`, so the build carries a working token + instead of a raw bucket credential. See "The `art` backend" below. + +Both land in the Garage `docker-cache` bucket on `s3.bigb.es`. Most of this +document is about the shape of the cache stages, which is the same either way; +where a rule differs, it says so. + +**Reference implementations:** `__SRHT_FAMILY__/sourcehut-coverage` and +`__SRHT_FAMILY__/sourcehut-tokens` for the overall shape, but take `cache_restore` +from `__SRHT_FAMILY__/sourcehut-bench` — the other two still carry a `go mod download +all` that dirties the tracked `go.sum` (see "Never `go mod download all`"). All +three keep the reasoning in `docs/ci.md` alongside the manifest; read +`docs/ci.md#cacher`, `#cache_restore`, `#never-go-mod-download-all` and +`#cache_save` before writing anything new. + +**For `art`:** `__SRHT_FAMILY__/sourcehut-artifacts` carries the switchable +`cache_setup` and `publish_client`, the task that publishes the binary every +other build installs — read `docs/ci.md#cache_setup` and `#publish_client` there. +`__SRHT_FAMILY__/bencher/.build.yml` on `license-free/v0.6.6` is the plain hardwired +form, and the only non-Alpine consumer. + +## Do not copy from the older siblings + +`sourcehut-spec`, `sourcehut-dolt` and `sourcehut-compare` still carry the +pre-v0.2.0 shape. Grepping `cache` across the family returns the stale form +three times and the current form twice, so a majority vote picks the wrong +donor. + +| stale | current | +|---|---| +| hand-written `curl` + `chmod` + `PATH` bootstrap | `curl -sSL …/ci-cacher/install.sh \| sh` | +| `if ! cacher dir download …; then …; cacher dir upload …; fi` | `cacher dir download --exec '