Let me confess a small heresy first, because it’s the reason any of this
happened. After a career spent as a branching man, gitflow, gitlabflow, a
tidy develop branch and a careful dance of merges, I’ve come round to
trunk-based development. I resisted it for years. It felt like working without
a net.
What changed my mind was working solo with an AI pair. The branch ceremony that earns its keep on a team of eight is just drag when it’s me and a model at two in the morning. So I’ve softened on “main is always deployable” and let the trunk act as the develop branch, with tagged releases as the actual source of truth. For compiled languages, where the artefact you ship is a built, tagged thing and not whatever’s on a server right now, that finally clicks.
I’d already rolled this out on my Go and Terraform projects with releaser-pleaser, a GitLab-native take on release-please: a bot keeps a Release MR open, and merging it cuts the tag. It’s the same model I wrote about when the infra repo moved to plan-on-merge, apply-on-tag. Lovely. Then I came to do the same for rust-tool-base, and Rust, being Rust, had opinions.
Rust brings its own toolchain
releaser-pleaser is happy to tag a repo and write a release. What it does not do
is cargo publish seventeen crates to crates.io in dependency order. Rust’s
release story isn’t “push a tag and let a runner build a binary”, it’s a whole
publishing pipeline with a public registry at the end of it, and that registry
has rules of its own. So for the Rust workspace I reached for the tool built for
exactly that job: release-plz. Same Release-MR shape,
but it understands cargo, versions every crate, and publishes the lot.
That was the right call. Getting it to actually do it was where I spent two days I’d quite like back.
The gauntlet before the gun
Before I got anywhere near the interesting failure, there was a run of CI
papercuts, the sort where every fix politely reveals the next one. GitLab checks
out a detached HEAD, and release-plz wants to be on a branch (“HEAD does not
point to a branch”), so you re-attach. Then the default CI_JOB_TOKEN can’t push
to a protected repo, so you point the remote at a real token. Then release-plz
assumes you’re on GitHub and errors that the repo “is not hosted in GitHub”, so
you tell it --forge gitlab. Then it refuses to run at all because the pages
job left a public/ directory lying about and the working tree is “dirty”, so
you stop pulling artefacts into the job.
Five merge requests before the thing would even start doing its actual job.
You can read the
scar tissue in the before_script;
every line in it is a fix for something on that list. None of it was hard.
It was just death by a thousand cuts, and I was feeling quite smug by the time
it finally reached the publish step.
I should not have been.
“Tag v0.5.1 already exists”
My release-plz.toml asked for
one tag per release:
git_tag_name = "v{{ version }}"
git_release_name = "v{{ version }}"
That felt obviously right. It matched the repo’s existing v0.5.0 tag, it’s how
a single-crate project tags, and the crates all share one workspace version
anyway. One version, one tag. What’s to argue with?
release-plz, that’s what. It tags per crate. So it publishes a crate, creates the tag, publishes the next crate, and tries to create the same tag again:
INFO published rtb-assets 0.5.1
ERROR failed to release package
Caused by:
0: failed to create git tag 'v0.5.1' with ref 'f6de975a75...'
1: Response body: { "message": "Tag v0.5.1 already exists" }
2: HTTP status client error (400 Bad Request) ... /repository/tags
The collision is annoying. What makes it a proper trap is the half-second before
it: published rtb-assets 0.5.1. That happened. On crates.io. For keeps. A
crates.io publish is forever, there is no unpublish, only a yank that still
leaves the name and version burned. So every time my flaky pipeline limped one
crate further and then fell over on the tag, it left another crate live on the
public registry that I could never take back. By the time the dust settled, six
of the seventeen were out there: rtb-assets and rtb-config, then on a later
retry rtb-credentials and rtb-error, then rtb-app and rtb-redact. Two
more permanent crates per failed run.
I assumed the default
My first fix was the clever one, and it deserves to be on display because it’s
the whole lesson in miniature. I deleted the git_tag_name line. My reasoning:
per-crate tags are release-plz’s native model, so surely its default does the
right thing without me spelling it out. I was confident enough to write it into
the commit message: “per-crate tags/releases (release-plz defaults).”
The next run collided on v0.5.1, exactly as before.
Because release-plz’s default git_tag_name is not per-crate. It’s the unified
v{{ version }}. I had deleted a line that said the wrong thing and replaced it
with a default that said the same wrong thing, then congratulated myself for
tidiness. If I’d spent thirty seconds on the configuration reference instead of
thirty seconds being clever, I’d have read that in black and white.
Same config, two answers
So I read the manual, and set it explicitly:
git_tag_name = "{{ package }}-v{{ version }}"
git_release_name = "{{ package }} v{{ version }}"
On my laptop, a dry run rendered exactly the per-crate tags I wanted. In CI, the
very next run published another crate and then created the tag v0.5.1. The
unified one. The wrong one. The one I had just, demonstrably, on main,
replaced.
Same release-plz.toml, two completely different answers depending on who was
asking.
That one took me an embarrassingly long time to see. release-plz does not read
your config from the working tree when it runs a release. It reads it from the
release commit, the commit that introduced the version it’s releasing. My
version was still 0.5.1, set days earlier on a commit that still carried the
unified template. You can see it in the failure: the tag it tries to create is
pinned to ref 'f6de975...', an old commit, not the HEAD that held my fix.
Every edit I made at the tip of main was real, committed, and utterly invisible
to the release of 0.5.1, because no version bump had created a fresh release
commit for it to read. My fix was correct and inert at the same time. The
dry run read my working directory and looked perfect; CI read history and did
something else.
There is no config change that rescues an in-flight release. The version was already out, half-published, tagged wrong, and pointed at a commit I couldn’t edit without bumping the version, which I couldn’t cleanly do with six crates already live.
Doing it the way I’d have done it a year ago
So I stopped. Three retries deep, each one a seventy-minute CI cycle thrown at an opaque mismatch, six crates already immovable on crates.io, and a tooling problem I now understood well enough to know the tool was never going to dig me out of this particular hole. The question quietly changed from “why is it doing this?” to “am I going to keep grinding, or finish this the way I would have before I had clever tooling?”
I went manual. cargo publish, the remaining eleven, by hand, in dependency
order: the leaf crates first and the rust-tool-base umbrella dead last,
because it depends on all of them. crates.io rate-limits new crate names, so
after a burst it simply made me wait, a roughly half-hour pause in the middle
while the registry caught its breath and I caught mine. Then one v0.5.1 tag,
cut by hand, and one GitLab release to match the convention. The next CI run came
up green, for the gloriously dull reason that there was nothing left to do:
every crate published, the tag already there.
Stop being clever and RTFM
The tool was never broken. Every single thing it did was documented behaviour I hadn’t bothered to read: that the default tag template is unified, that the model is per-crate, that a release reads its config from the release commit and not from HEAD. I assumed my way past the manual three times in a row, and each assumption cost me real, permanent state on a public registry that doesn’t take returns.
And that’s the part that actually stung, because I should have known better than most. I wasn’t a beginner here. I knew the Release-MR pattern cold, I’d shipped it half a dozen times with releaser-pleaser on my Go and Terraform repos. That familiarity was the trap. I trusted the pattern and skipped the tool, on the lazy assumption that something I understood well in one tool would behave the same in the next. release-plz carries the same design, but it’s a different tool, with its own defaults and its own idea of where the config lives. The pattern came across fine. The mechanics didn’t, and I never thought to check.
So here’s the lesson, written down in the hope it sticks this time: no matter how familiar I am with a pattern or a design, the moment I switch the tool that implements it, reading the manual is paramount. The familiarity is exactly what tempts you to skip it, and exactly why you can’t. (The narrower, more practical one, while I’m here: a config change that affects how a release behaves has to travel with a version bump, or it sits there looking applied and doing nothing.)
release-plz is genuinely good, and every release since has gone out clean on the first try, the way the rest of the CI now does. I just had to stop being clever long enough to read how it actually works. RTFM. I’ll get it tattooed eventually.
