A signing key that never leaves KMS

Thu, 11 Jun 2026 00:00:00 +0000

The last post in this series walked through how a tool verifies a release signature the platform can’t forge. That post had a loose end dangling off the back of it, and I knew it the whole time I was writing. Because a signature has to be produced by a private key… and a private signing key is the single worst thing in this entire story to lose. Steal it, and you sign malware that sails through every check I spent two posts building, signature and all. So where does that key live? The answer I landed on is the one this whole post is about: inside AWS KMS, and it never comes out.

The only key you can’t steal

Think about where a signing key normally ends up. A file on a build server. A secret in CI. A key on the release engineer’s laptop, “just for the release, I’ll delete it after”. Every one of those is a copy, and every copy is one more thing somebody can read, exfiltrate, or quietly clone while your back is turned. You can wrap them in passphrases and vaults and rotation policies all you like, and you’re still standing guard over a thing that exists in a place you don’t fully control.

The way out is almost annoyingly simple to state: the only key nobody can steal is the one that was never anywhere to be stolen from. So don’t hold the key at all. Let something else hold it, somewhere it has no export path, and ask that thing to sign for you.

That thing is AWS KMS. This is the infrastructure side of the question I opened the signing series with, finally answered with real Terraform.

A key that’s born in the box and stays there

The signing key is an asymmetric KMS key, and the module that provisions it is small enough to read in one sitting:

resource "aws_kms_key" "this" {
 description = var.description
 key_usage = "SIGN_VERIFY"
 customer_master_key_spec = var.key_spec # default RSA_4096

 # Asymmetric SIGN_VERIFY keys do not support KMS-managed rotation;
 # rotation is handled by minting a new key (alias = `<name>-v2`) and
 # publishing the v2 public key alongside the v1 key (dual-sign window).
 enable_key_rotation = false

 policy = data.aws_iam_policy_document.key_policy.json
}

The private half of that key is generated inside KMS and there is no API that hands it back to you. You don’t sign with it the way you’d sign with a file. You call kms:Sign: the bytes you want signed go up, a signature comes back down, and the key itself never moves. An attacker who completely owns my CI, my account, my laptop, can ask KMS to sign things for as long as their access lasts… but they can’t walk off with the key and keep signing forever. The blast radius is “while I’m compromised”, not “until I rotate a key I didn’t know had leaked three years ago”.

Why RSA-4096 and not the Ed25519 I’d normally reach for? Because KMS asymmetric signing doesn’t offer Ed25519, and OpenPGP’s packet format is tied to the algorithm that signed it, so the choice of key spec ripples all the way out to the signature on the wire. RSA-4096 is the strong option KMS does offer, so RSA-4096 is what the workflow is built around. A constraint of the box shaped the cryptography, not the other way round, and I’d rather say so than pretend I picked RSA on purpose.

Minting an OpenPGP key from a key you can’t hold

Here’s the part I find genuinely neat. OpenPGP wants a private key to self-sign its own public key when you generate it. And I don’t have a private key in any form I can hand to a library… it’s sitting in KMS, behind a door with no handle on my side. So how do you produce a valid OpenPGP public key at all?

go-tool-base leans on a small Go interface, crypto.Signer: anything that can return its public key and sign a digest. A KMS-backed signer satisfies it by turning each Sign call into a kms:Sign request. Then pkg/openpgpkey builds the OpenPGP entity around that signer:

func Entity(signer crypto.Signer, name, email string, creationTime time.Time) (*openpgp.Entity, error) {
	rsaPub, ok := signer.Public().(*rsa.PublicKey)
	// ...
	pubPkt := packet.NewRSAPublicKey(creationTime, rsaPub)
	// Construct the private-key packet directly (rather than
	// packet.NewSignerPrivateKey, which panics on opaque signers):
	// the crypto.Signer drives the actual signing, so a KMS-backed
	// signer works here.
	privPkt := &packet.PrivateKey{PublicKey: *pubPkt, PrivateKey: signer}
	// ...
}

Look at that PrivateKey packet. The field where OpenPGP expects the secret key material holds the crypto.Signer instead, which is to say, a remote handle to KMS. When the entity self-signs its public key, that self-signature is computed by KMS. gtb keys mint runs exactly this and writes out an ASCII-armored OpenPGP public key, and at no point did a single byte of private key material exist on the machine that minted it. The OpenPGP “private key” is a phone line to a vault, not a key.

That public key is what gets published off-platform over WKD and baked into the binary, the two trust anchors that post cross-checks.

Access without a human and without a standing key

A key that never leaves KMS is only as good as the rules about who may call kms:Sign. The signer role is deliberately narrow: it can call kms:Sign and kms:GetPublicKey on this one key and nothing else, and it is assumable only over OIDC from specific CI subjects, the same keyless federation the rest of the estate runs on. No human holds it. No long-lived access key sits in a CI variable waiting to leak. A release job federates in for its few minutes, signs, and the credentials evaporate with the runner.

So the chain of “who can sign a release” has no standing secret in it anywhere. Not a key file, not an access key, not a console user. Just a short-lived token, scoped to two API calls, on a key that can’t be exported.

The real cost: rotation is manual

This isn’t free, and the bit it taxes you on is rotation. KMS won’t auto-rotate an asymmetric SIGN_VERIFY key, which is why the module sets enable_key_rotation = false rather than leaving a default on. Rotating means minting a new key (a -v2 alias), publishing its public key alongside the old one, and running a dual-sign window long enough that clients have picked up the new anchor before you retire the old. It’s manual, it’s a runbook, and pretending otherwise would be the kind of thing this series exists to argue against. The trade I made was: a key with no exfiltration path, in exchange for rotation I have to do by hand. For a release-signing key, that’s the right side of the trade.

Why this is a command and not a script I hid

The origin of all this is a good deal less tidy than the result. I was working through the key-generation runbook, creating the offline rotation key with a gpg command I’d copied straight off my own page… and it just hung. No error, no prompt, just a cursor blinking while gpg waited on something it never bothered to mention.

My first instinct was the lazy one: drop the minting script into a scripts folder in my infra repo and never speak of it again. Then it nagged. That repo’s private, so the recipe would live somewhere nobody else could ever reach, and I’d already half-promised myself a tutorial walking people through this exact setup. So it shouldn’t sit in infra at all. It should be a gtb command, with a pluggable backend so anyone can swap my KMS for whatever provider they happen to run.

The deeper objection is the one that actually shaped it, though. I didn’t want to be shelling out to gpg by hand in the first place. gtb is a tool I hand to other people, and every time it drops to the shell for some gpg incantation, that’s an environment I’m asking the next person to reproduce, a dependency to install, a fiddly step to get subtly wrong, all before they can sign a single thing. The aim was to keep as much of this inside the box as I could: mint the key, build the WKD tree, produce the signature, all in pure Go, with no gpg on the path and no gpg-wks-client either.

So gtb keys mint pulls the public half out of your KMS key and frames it as OpenPGP, the trick from earlier; gtb keys wkd builds the tree ready to upload; and gtb sign produces the detached signature through that same remote round-trip. What comes out is an entirely ordinary OpenPGP signature gpg --verify is happy with, so you’re not locked into anything of mine. And none of it is just for me: build your tool on go-tool-base and the same handful of commands stands you up with this exact model, pointed at your own KMS. No cloud KMS to hand? There’s a local backend, a plain key on disk, to wire the whole thing together on your laptop first. These are commands for you, the person shipping the tool. Your users never run mytool keys mint… they just get updates that quietly check themselves, which was the whole idea two posts ago.

That setup deserves a walkthrough of its own, and it’ll get one. For now, the ergonomics were the point, not a nicety bolted on afterwards. The safest setup in the world is no use to anyone if it takes a PhD to stand up.

Where this leaves the whole story

Step back and the full loop is finally closed. The private key is born in KMS and never leaves it. Its public key is minted from it, with KMS computing its own self-signature. That public key is published off-platform and embedded in the binary. Releases are signed by KMS, reached only through short-lived OIDC federation. And the client verifies against the embedded and WKD keys cross-checked against each other. At no single point in that chain is there a thing an attacker can grab that lets them forge a release, and the most dangerous thing of all, the private key, has no theft path because it has no export path.

That’s the thread running through the whole signing series, from the very first checksum to here: the strongest control isn’t a better lock on the key. It’s arranging things so the key was never somewhere you could lose it. Nobody is coming to clean your supply chain, so the least you can do is leave it nothing worth stealing.

Aws-Kms on PHP Boy Scout