<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>AWS on PHP Boy Scout</title><link>https://blog-570662.gitlab.io/tags/aws/</link><description>Recent content in AWS on PHP Boy Scout</description><generator>Hugo -- gohugo.io</generator><language>en-gb</language><copyright>Matt Cockayne</copyright><lastBuildDate>Thu, 14 May 2026 00:00:00 +0000</lastBuildDate><atom:link href="https://blog-570662.gitlab.io/tags/aws/index.xml" rel="self" type="application/rss+xml"/><item><title>A 403 you can't fix in IAM</title><link>https://blog-570662.gitlab.io/a-403-you-cant-fix-in-iam/</link><pubDate>Thu, 14 May 2026 00:00:00 +0000</pubDate><guid>https://blog-570662.gitlab.io/a-403-you-cant-fix-in-iam/</guid><description>&lt;img src="https://blog-570662.gitlab.io/a-403-you-cant-fix-in-iam/cover-a-403-you-cant-fix-in-iam.png" alt="Featured image of post A 403 you can't fix in IAM" /&gt;&lt;p&gt;&lt;a class="link" href="https://blog-570662.gitlab.io/no-access-keys-in-ci/" &gt;The OIDC post&lt;/a&gt; explained the handshake that lets a GitLab pipeline deploy to AWS with no stored key. This is the story of the first time I got it wrong, and spent an afternoon fixing the wrong thing. The error was a flat 403 from AWS, and the maddening part is that no amount of editing the IAM policy was ever going to fix it.&lt;/p&gt;
&lt;h2 id="a-403-on-the-first-real-run"&gt;A 403 on the first real run
&lt;/h2&gt;&lt;p&gt;The OIDC post covered the handshake: GitLab CI mints a signed token, AWS exchanges it for short-lived credentials against a role whose trust policy names the pipeline. During the GitLab migration I wired exactly that up for the &lt;code&gt;infra&lt;/code&gt; repo, including a trust policy condition meant to let merge-request pipelines run a plan.&lt;/p&gt;
&lt;p&gt;The first merge request that should have triggered &lt;code&gt;tofu-plan&lt;/code&gt; didn&amp;rsquo;t run it. The job failed, and the error from AWS was a flat &lt;code&gt;AccessDenied&lt;/code&gt;. A 403.&lt;/p&gt;
&lt;h2 id="the-instinct-and-why-it-wastes-an-afternoon"&gt;The instinct, and why it wastes an afternoon
&lt;/h2&gt;&lt;p&gt;The instinct on an IAM 403 is immediate and almost always right: the policy&amp;rsquo;s wrong, so go and edit the policy. Tighten the condition. Loosen the condition. Check the wildcard. Re-read the &lt;code&gt;sub&lt;/code&gt; pattern character by character.&lt;/p&gt;
&lt;p&gt;All of that was wasted, and it was wasted for a reason that took me far too long to see. The trust policy wasn&amp;rsquo;t matching the &lt;em&gt;wrong&lt;/em&gt; value. It was matching a value that &lt;em&gt;does not exist&lt;/em&gt;. No amount of editing a condition makes it match a thing that&amp;rsquo;s never present.&lt;/p&gt;
&lt;h2 id="what-is-actually-in-the-token"&gt;What is actually in the token
&lt;/h2&gt;&lt;p&gt;GitLab&amp;rsquo;s OIDC token has a &lt;code&gt;sub&lt;/code&gt; claim that encodes the pipeline&amp;rsquo;s context, and part of that encoding is a &lt;code&gt;ref_type&lt;/code&gt;. I&amp;rsquo;d assumed &lt;code&gt;ref_type&lt;/code&gt; could be &lt;code&gt;branch&lt;/code&gt;, &lt;code&gt;tag&lt;/code&gt;, or &lt;code&gt;mr&lt;/code&gt;, because a pipeline can certainly be a branch pipeline, a tag pipeline, or a merge-request pipeline. So the trust policy, for the plan job, matched a &lt;code&gt;sub&lt;/code&gt; containing &lt;code&gt;ref_type:mr&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;That assumption was wrong. GitLab&amp;rsquo;s &lt;code&gt;ref_type&lt;/code&gt; is &lt;code&gt;branch&lt;/code&gt; or &lt;code&gt;tag&lt;/code&gt;. That&amp;rsquo;s the entire set. There is no &lt;code&gt;mr&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;A merge-request pipeline doesn&amp;rsquo;t run against a merge-request ref. It runs against the source &lt;em&gt;branch&lt;/em&gt;. So its token&amp;rsquo;s &lt;code&gt;sub&lt;/code&gt; carries &lt;code&gt;ref_type:branch&lt;/code&gt;, like any other branch pipeline. The trust policy condition asked for &lt;code&gt;ref_type:mr&lt;/code&gt;, GitLab never puts &lt;code&gt;mr&lt;/code&gt; in a token, the condition was therefore never true, and every merge-request pipeline got a 403. Forever, until the policy stopped asking for a claim that isn&amp;rsquo;t real.&lt;/p&gt;
&lt;h2 id="the-fix-and-the-lesson-worth-more-than-the-fix"&gt;The fix, and the lesson worth more than the fix
&lt;/h2&gt;&lt;p&gt;The fix is small once it&amp;rsquo;s visible: match &lt;code&gt;ref_type:branch&lt;/code&gt; and narrow it down by branch name or project path instead. An afternoon of policy edits, and the actual change is one word.&lt;/p&gt;
&lt;p&gt;The lesson is the part worth keeping. When an OIDC trust fails, the useful question is never &amp;ldquo;is my policy clever enough&amp;rdquo;. It&amp;rsquo;s &amp;ldquo;what&amp;rsquo;s &lt;em&gt;actually in the token&lt;/em&gt;&amp;rdquo;. An OIDC trust policy can only ever match the claims the identity provider genuinely asserts, and the gap between what a provider asserts and what you &lt;em&gt;assumed&lt;/em&gt; it asserts is precisely where this class of bug lives.&lt;/p&gt;
&lt;p&gt;So the move, when an OIDC handshake 403s, is to get hold of a real token and decode it. Look at the actual &lt;code&gt;sub&lt;/code&gt;, the actual claims, the actual values. Match what&amp;rsquo;s there. A 403 that survives every sensible edit to the policy is usually not a policy that&amp;rsquo;s too loose or too strict. It&amp;rsquo;s a policy matching a claim that was never going to be in the token.&lt;/p&gt;
&lt;h2 id="the-habit-it-left-behind"&gt;The habit it left behind
&lt;/h2&gt;&lt;p&gt;I wired an OIDC trust policy to let merge-request pipelines plan, by matching a &lt;code&gt;sub&lt;/code&gt; claim with &lt;code&gt;ref_type:mr&lt;/code&gt;. The first real merge request got a 403, and no edit to the policy fixed it, because GitLab&amp;rsquo;s &lt;code&gt;ref_type&lt;/code&gt; is only ever &lt;code&gt;branch&lt;/code&gt; or &lt;code&gt;tag&lt;/code&gt;. A merge-request pipeline runs on a branch ref, so the &lt;code&gt;mr&lt;/code&gt; value the policy demanded was never in any token.&lt;/p&gt;
&lt;p&gt;The fix was one word. The habit it left behind is the valuable bit: when an OIDC trust fails, stop editing the policy and go and read a real token. A trust policy can only match what the provider actually asserts, and &amp;ldquo;what I assumed it asserts&amp;rdquo; is where the 403 was hiding the whole time. (If this shape of bug feels familiar by the end of the series, that&amp;rsquo;s not an accident: I &lt;a class="link" href="https://blog-570662.gitlab.io/two-bugs-that-taught-me-the-rules/" &gt;come back to it&lt;/a&gt; with two more from exactly the same family.)&lt;/p&gt;</description></item><item><title>Routing security findings without the noise</title><link>https://blog-570662.gitlab.io/routing-security-findings-without-the-noise/</link><pubDate>Tue, 12 May 2026 00:00:00 +0000</pubDate><guid>https://blog-570662.gitlab.io/routing-security-findings-without-the-noise/</guid><description>&lt;img src="https://blog-570662.gitlab.io/routing-security-findings-without-the-noise/cover-routing-security-findings-without-the-noise.png" alt="Featured image of post Routing security findings without the noise" /&gt;&lt;p&gt;Turning on GuardDuty and Security Hub gives you threat detection. It also gives you a firehose. And an alert system that dutifully forwards everything in that firehose isn&amp;rsquo;t monitoring, it&amp;rsquo;s a very efficient way of training your team to ignore alerts. So the &lt;code&gt;alerts&lt;/code&gt; module&amp;rsquo;s real job isn&amp;rsquo;t detection at all. It&amp;rsquo;s deciding what&amp;rsquo;s actually worth interrupting a human for, and the interesting part is everything it deliberately throws away.&lt;/p&gt;
&lt;h2 id="detection-is-the-easy-half"&gt;Detection is the easy half
&lt;/h2&gt;&lt;p&gt;Switching on threat detection in an AWS account is a few resources. GuardDuty, Security Hub with its standards, IAM Access Analyzer: &lt;a class="link" href="https://blog-570662.gitlab.io/hardening-the-account-that-will-hold-the-keys/" &gt;the security baseline&lt;/a&gt; does exactly that. From then on, the account is generating findings.&lt;/p&gt;
&lt;p&gt;And it generates a lot of them. Plenty are low-severity, informational, or simply the normal texture of a cloud account. If you wire every finding to an email or a pager, you haven&amp;rsquo;t built monitoring. You&amp;rsquo;ve built noise. And noise has a specific failure mode: people stop reading it, and the one finding that genuinely mattered scrolls past unread alongside two hundred that didn&amp;rsquo;t.&lt;/p&gt;
&lt;p&gt;So the valuable work isn&amp;rsquo;t detection. It&amp;rsquo;s &lt;em&gt;routing&lt;/em&gt;: deciding what&amp;rsquo;s worth interrupting a human for, and letting the rest sit quietly in a console for whenever someone reviews it.&lt;/p&gt;
&lt;h2 id="forward-the-severe-leave-the-rest"&gt;Forward the severe, leave the rest
&lt;/h2&gt;&lt;p&gt;The &lt;code&gt;alerts&lt;/code&gt; module routes findings with EventBridge rules into an SNS topic that emails out. The rules are deliberately picky. GuardDuty findings are forwarded only at severity 7 and above. Security Hub findings are forwarded only at HIGH and CRITICAL.&lt;/p&gt;
&lt;p&gt;Everything below those thresholds isn&amp;rsquo;t discarded. It&amp;rsquo;s still in GuardDuty and Security Hub, where someone doing a review will see it. It just doesn&amp;rsquo;t get to interrupt anyone&amp;rsquo;s day. The threshold is the line between &amp;ldquo;look at this now&amp;rdquo; and &amp;ldquo;look at this sometime&amp;rdquo;.&lt;/p&gt;
&lt;h2 id="the-duplicate-you-would-otherwise-send-twice"&gt;The duplicate you would otherwise send twice
&lt;/h2&gt;&lt;p&gt;Here&amp;rsquo;s the subtle one, and it&amp;rsquo;s the kind of thing you only find by looking closely at where findings come from.&lt;/p&gt;
&lt;p&gt;Security Hub is an aggregator. It pulls findings &lt;em&gt;in&lt;/em&gt; from other services, GuardDuty among them. So a single GuardDuty finding can show up in two places: in GuardDuty itself, and again in Security Hub as an aggregated copy.&lt;/p&gt;
&lt;p&gt;A rule on GuardDuty findings and a rule on Security Hub HIGH/CRITICAL findings would therefore both fire for the same underlying GuardDuty finding. One event, two emails. Do that across an account and a meaningful fraction of your alert volume is just the same findings counted twice, which is its own kind of noise.&lt;/p&gt;
&lt;p&gt;So the Security Hub rule explicitly excludes findings whose &lt;code&gt;ProductName&lt;/code&gt; is GuardDuty, with an &lt;code&gt;anything-but&lt;/code&gt; match. GuardDuty findings come through the GuardDuty rule. The Security Hub rule handles everything Security Hub adds that GuardDuty didn&amp;rsquo;t already report. One finding, one alert, regardless of how many services it passed through.&lt;/p&gt;
&lt;h2 id="two-tripwires-on-the-root-account"&gt;Two tripwires on the root account
&lt;/h2&gt;&lt;p&gt;Findings are about threats the detectors recognise. The module adds two alarms about something simpler: the root account doing anything at all.&lt;/p&gt;
&lt;p&gt;One CloudWatch alarm fires on a root console sign-in. The other fires on any root API call that isn&amp;rsquo;t a console login. In a well-run AWS account, the root user does almost nothing after initial setup: day-to-day work happens through roles. So root activity isn&amp;rsquo;t a &amp;ldquo;finding&amp;rdquo; to be assessed for severity. It&amp;rsquo;s a tripwire. Any of it, in an account that should be silent, is worth an immediate look, and the two alarms say so directly.&lt;/p&gt;
&lt;h2 id="why-a-quiet-alert-stream-matters-here"&gt;Why a quiet alert stream matters here
&lt;/h2&gt;&lt;p&gt;This is monitoring for the account that&amp;rsquo;s going to hold the release-signing key, and that raises the stakes on getting the routing right.&lt;/p&gt;
&lt;p&gt;If a key-bearing account ever does come under attack, the alert that says so has to be &lt;em&gt;seen&lt;/em&gt;. An alert stream that&amp;rsquo;s mostly noise and duplicates is, functionally, no alerting at all, because the people who&amp;rsquo;d act on it have long since tuned it out. Routing the stream down to &amp;ldquo;severe, deduplicated, plus root tripwires&amp;rdquo; is what keeps it something a human will still read on the day it finally matters.&lt;/p&gt;
&lt;h2 id="the-short-version"&gt;The short version
&lt;/h2&gt;&lt;p&gt;GuardDuty and Security Hub make detection easy. The hard, valuable part is routing: forwarding what deserves to interrupt someone and leaving the rest in a console.&lt;/p&gt;
&lt;p&gt;The &lt;code&gt;alerts&lt;/code&gt; module forwards GuardDuty at severity 7-plus and Security Hub at HIGH/CRITICAL, and it drops the duplicate that aggregation creates by excluding GuardDuty-sourced findings from the Security Hub rule, so one finding is one alert. Two CloudWatch alarms act as tripwires on root-account activity, which should be near-zero. For the account that will hold the signing key, a quiet, trustworthy alert stream isn&amp;rsquo;t a nicety. It&amp;rsquo;s the difference between monitoring and theatre.&lt;/p&gt;</description></item><item><title>Hardening the account that will hold the keys</title><link>https://blog-570662.gitlab.io/hardening-the-account-that-will-hold-the-keys/</link><pubDate>Sat, 09 May 2026 00:00:00 +0000</pubDate><guid>https://blog-570662.gitlab.io/hardening-the-account-that-will-hold-the-keys/</guid><description>&lt;img src="https://blog-570662.gitlab.io/hardening-the-account-that-will-hold-the-keys/cover-hardening-the-account-that-will-hold-the-keys.png" alt="Featured image of post Hardening the account that will hold the keys" /&gt;&lt;p&gt;&lt;a class="link" href="https://blog-570662.gitlab.io/the-bootstrap-that-does-almost-nothing/" &gt;Bootstrapping the account&lt;/a&gt; got it &lt;em&gt;ready&lt;/em&gt;: somewhere to store state, an identity to deploy as, enough for the next &lt;code&gt;tofu apply&lt;/code&gt; to run. Ready is not the same as safe. An account with no audit trail, nothing watching it, and no considered way for a human to get in is fine for experimenting and absolutely not where you put the most sensitive key in the system. So before the signing key goes anywhere near it, the account gets a security baseline.&lt;/p&gt;
&lt;h2 id="ready-is-not-the-same-as-safe"&gt;Ready is not the same as safe
&lt;/h2&gt;&lt;p&gt;The bootstrap post ended with an account that was &lt;em&gt;ready&lt;/em&gt;: it had somewhere to store state and a CI identity to deploy as. The next &lt;code&gt;tofu apply&lt;/code&gt; could run.&lt;/p&gt;
&lt;p&gt;Ready is not safe. That account still has no audit trail, so nobody could tell you afterwards what happened in it. It has no threat detection, so nothing is watching. Its defaults are AWS&amp;rsquo;s defaults, which are not a security posture. There&amp;rsquo;s no considered way for a human to get in. An account in that condition is fine for experimenting. It&amp;rsquo;s not somewhere you put the most sensitive key in the whole system.&lt;/p&gt;
&lt;p&gt;So before the signing key is anywhere near it, the account gets a security baseline.&lt;/p&gt;
&lt;h2 id="the-baseline-in-one-downstream-stack"&gt;The baseline, in one downstream stack
&lt;/h2&gt;&lt;p&gt;&lt;code&gt;terraform-aws-security-baseline&lt;/code&gt; is that baseline, and it&amp;rsquo;s exactly the downstream stack the bootstrap post promised: applied &lt;em&gt;through&lt;/em&gt; the automation role bootstrap created, not bootstrapped specially.&lt;/p&gt;
&lt;p&gt;It&amp;rsquo;s six sub-modules, each behind an &lt;code&gt;enable_*&lt;/code&gt; toggle: &lt;code&gt;account-hardening&lt;/code&gt; (IAM password policy, account-wide S3 public-access blocking, default EBS encryption), &lt;code&gt;audit-logging&lt;/code&gt; (a multi-region CloudTrail with log-file validation), &lt;code&gt;aws-config&lt;/code&gt;, &lt;code&gt;threat-detection&lt;/code&gt; (GuardDuty, Security Hub, IAM Access Analyzer), &lt;code&gt;alerts&lt;/code&gt;, and &lt;code&gt;operator-role&lt;/code&gt;. Together they turn a bare account into one that records what happens, watches for trouble, and controls who gets in.&lt;/p&gt;
&lt;p&gt;Most of those are the expected baseline. The operator role is the one worth slowing down on, because it&amp;rsquo;s built backwards from how people usually think about an admin role.&lt;/p&gt;
&lt;h2 id="the-operator-role-and-the-inversion"&gt;The operator role, and the inversion
&lt;/h2&gt;&lt;p&gt;&lt;code&gt;InfraAdmin&lt;/code&gt; is the human way into the account: the role a person assumes to do operator work. Two things define it.&lt;/p&gt;
&lt;p&gt;The trust policy decides &lt;em&gt;who&lt;/em&gt; may assume it. It trusts only the account root principal, and it requires multi-factor authentication: the assume call must carry &lt;code&gt;aws:MultiFactorAuthPresent&lt;/code&gt;, and &lt;code&gt;aws:MultiFactorAuthAge&lt;/code&gt; bounds how recently that MFA was performed. No MFA, no role. So far this is a careful but ordinary admin role.&lt;/p&gt;
&lt;p&gt;The inversion is a &lt;em&gt;second&lt;/em&gt;, separate inline policy, and it&amp;rsquo;s almost entirely &lt;code&gt;Deny&lt;/code&gt;. It denies, using &lt;code&gt;NotAction&lt;/code&gt;, anything where &lt;code&gt;aws:RequestedRegion&lt;/code&gt; falls outside an allowed set of regions. The role&amp;rsquo;s &lt;em&gt;power&lt;/em&gt; comes from an admin grant. This inline policy &lt;em&gt;fences&lt;/em&gt; that power.&lt;/p&gt;
&lt;p&gt;That&amp;rsquo;s the part worth holding onto. People picture an admin role as a list of what it can do. This one is better understood by what it &lt;em&gt;cannot&lt;/em&gt;: it cannot act outside its permitted regions, full stop. A fat-fingered command, or a compromised session, cannot quietly spin resources up in some region nobody&amp;rsquo;s watching. The fence is as much the point of the role as the grant is.&lt;/p&gt;
&lt;h2 id="the-carve-out-because-honesty"&gt;The carve-out, because honesty
&lt;/h2&gt;&lt;p&gt;There&amp;rsquo;s a fiddly detail, and it&amp;rsquo;s the kind of thing that makes the region fence real rather than theoretical.&lt;/p&gt;
&lt;p&gt;Some AWS services are global. IAM, CloudFront, Route 53 and friends have no region, and they don&amp;rsquo;t honour &lt;code&gt;aws:RequestedRegion&lt;/code&gt;. A naive region-deny would therefore deny calls to IAM, and you&amp;rsquo;d lock yourself out of the very service you manage access with. (A close cousin of the kind of self-inflicted lockout I&amp;rsquo;ll come back to in a &lt;a class="link" href="https://blog-570662.gitlab.io/a-403-you-cant-fix-in-iam/" &gt;later post&lt;/a&gt;.)&lt;/p&gt;
&lt;p&gt;So the Deny carries explicit carve-outs for the global services. It isn&amp;rsquo;t elegant, and it can&amp;rsquo;t be: the global-versus-regional split is just a fact of AWS, and a correct region fence has to account for it. The carve-out list is the honest cost of the control working.&lt;/p&gt;
&lt;h2 id="harden-the-room-then-move-the-keys-in"&gt;Harden the room, then move the keys in
&lt;/h2&gt;&lt;p&gt;There&amp;rsquo;s an order to all of this, and the order is the argument.&lt;/p&gt;
&lt;p&gt;The account that will hold the signing key has to be audited before the key arrives, so that from day one every call against it is in CloudTrail. It has to be watched before the key arrives, so GuardDuty is already looking. It has to be access-controlled before the key arrives, so the only human path in is MFA-gated and region-fenced.&lt;/p&gt;
&lt;p&gt;You don&amp;rsquo;t move something valuable into a room and then think about locks. You build the room, fit the locks, check they work, and &lt;em&gt;then&lt;/em&gt; move the valuable thing in. The security baseline is fitting the locks. The signing key comes later, into a room already built for it.&lt;/p&gt;
&lt;h2 id="worth-remembering"&gt;Worth remembering
&lt;/h2&gt;&lt;p&gt;Bootstrapping an account makes it ready for the next deploy. It does not make it safe to hold anything that matters. &lt;code&gt;terraform-aws-security-baseline&lt;/code&gt; is the downstream stack that closes that gap: audit logging, AWS Config, threat detection, account hardening, and an operator role, applied through the CI role bootstrap created.&lt;/p&gt;
&lt;p&gt;The operator role is the piece to study. It&amp;rsquo;s MFA-gated on the way in, and then fenced by a separate, almost-all-&lt;code&gt;Deny&lt;/code&gt; inline policy that confines it to permitted regions, with carve-outs for the global services that have no region. An admin role defined as much by its fence as its grant. Harden the room first; the keys move in afterwards.&lt;/p&gt;</description></item><item><title>No access keys in CI</title><link>https://blog-570662.gitlab.io/no-access-keys-in-ci/</link><pubDate>Fri, 08 May 2026 00:00:00 +0000</pubDate><guid>https://blog-570662.gitlab.io/no-access-keys-in-ci/</guid><description>&lt;img src="https://blog-570662.gitlab.io/no-access-keys-in-ci/cover-no-access-keys-in-ci.png" alt="Featured image of post No access keys in CI" /&gt;&lt;p&gt;A long-lived AWS access key, sitting in a CI system, is just about the single credential I&amp;rsquo;d most like to be rid of. It&amp;rsquo;s powerful, it never expires unless someone remembers to rotate it (nobody remembers to rotate it), and it lives in one of the most attractive targets in the whole supply chain. For infrastructure that&amp;rsquo;s eventually going to hold a release-signing key, it&amp;rsquo;s exactly the wrong place to start. So the &lt;code&gt;phpboyscout&lt;/code&gt; infrastructure has no AWS access key in CI at all. None.&lt;/p&gt;
&lt;h2 id="the-access-key-you-dont-want"&gt;The access key you don&amp;rsquo;t want
&lt;/h2&gt;&lt;p&gt;A CI pipeline that runs &lt;code&gt;tofu apply&lt;/code&gt; against AWS needs AWS credentials. The traditional way to give it some is an IAM user with an access key pair, pasted into the CI system as a masked variable.&lt;/p&gt;
&lt;p&gt;Look at what that key is. It&amp;rsquo;s long-lived: it works until someone remembers to rotate it, and rotating it is a chore, so mostly nobody does. It&amp;rsquo;s powerful: it can apply infrastructure, so it can do nearly anything. And it&amp;rsquo;s sitting in a CI system, which is one of the most attractive targets in your whole supply chain. You&amp;rsquo;ve taken your highest-value credential and stored a permanent copy of it in a place built for running automated jobs.&lt;/p&gt;
&lt;p&gt;For infrastructure that&amp;rsquo;s going to hold a release-signing key, that&amp;rsquo;s precisely the wrong starting point. So the &lt;code&gt;phpboyscout&lt;/code&gt; infrastructure has no AWS access key in CI at all. Not a well-guarded one. None.&lt;/p&gt;
&lt;h2 id="federation-instead-of-a-stored-secret"&gt;Federation instead of a stored secret
&lt;/h2&gt;&lt;p&gt;The replacement is OIDC federation, and the shape of it is worth walking through, because it&amp;rsquo;s genuinely different from &amp;ldquo;a secret, but better&amp;rdquo;.&lt;/p&gt;
&lt;p&gt;A modern CI platform can mint an OIDC token. GitLab does this with an &lt;code&gt;id_tokens:&lt;/code&gt; block: at job time, GitLab issues a short-lived JSON Web Token, signed by GitLab, that asserts a set of facts. This is project X. This is pipeline Y. This is running on ref Z, of this type.&lt;/p&gt;
&lt;p&gt;AWS can consume that. The &lt;code&gt;sts:AssumeRoleWithWebIdentity&lt;/code&gt; call takes such a token and, if it satisfies an IAM role&amp;rsquo;s trust policy, returns short-lived AWS credentials for that role. The trust policy is where the control lives: it names GitLab as a trusted token issuer, and it constrains the token&amp;rsquo;s &lt;code&gt;sub&lt;/code&gt; claim so that only the specific project, and the specific refs, you intend can assume the role.&lt;/p&gt;
&lt;p&gt;Put it together: the pipeline asks GitLab for a token, hands it to AWS, and gets back credentials that last about an hour and are scoped to one role. Nothing long-lived is stored anywhere. The credential exists only for the job that needs it, and it can&amp;rsquo;t be stolen from a CI variable store, because it was never in one.&lt;/p&gt;
&lt;h2 id="two-halves-of-one-handshake"&gt;Two halves of one handshake
&lt;/h2&gt;&lt;p&gt;That handshake is built by two of the repos in this series, each owning one side.&lt;/p&gt;
&lt;p&gt;&lt;a class="link" href="https://blog-570662.gitlab.io/the-bootstrap-that-does-almost-nothing/" &gt;&lt;code&gt;terraform-aws-bootstrap&lt;/code&gt;&lt;/a&gt; builds the AWS half, in its &lt;code&gt;automation-iam&lt;/code&gt; module: it registers GitLab as an OIDC identity provider in the account, and it creates the automation role with the trust policy that decides which pipelines may assume it.&lt;/p&gt;
&lt;p&gt;The CI components build the consuming half: the &lt;code&gt;id_tokens:&lt;/code&gt; block that asks GitLab for the JWT, and then simply letting the AWS provider&amp;rsquo;s own credential chain perform the exchange. The pipeline doesn&amp;rsquo;t call &lt;code&gt;sts&lt;/code&gt; by hand. It presents the token; the SDK does the rest.&lt;/p&gt;
&lt;h2 id="the-gotcha-dont-set-a-profile"&gt;The gotcha: don&amp;rsquo;t set a profile
&lt;/h2&gt;&lt;p&gt;There&amp;rsquo;s one quiet way to break this, and a stack can look completely correct while doing it.&lt;/p&gt;
&lt;p&gt;The AWS SDK finds credentials by walking a chain of sources in order. The web-identity path, the one that uses the OIDC token, is one link in that chain. It triggers off environment variables the CI sets up automatically.&lt;/p&gt;
&lt;p&gt;But if the &lt;code&gt;aws&lt;/code&gt; provider block has a hardcoded &lt;code&gt;profile = &amp;quot;...&amp;quot;&lt;/code&gt;, the SDK takes the &lt;em&gt;profile&lt;/em&gt; link of the chain instead, and never reaches the web-identity link. A &lt;code&gt;profile&lt;/code&gt; line is the sort of thing that ends up in a provider block from someone&amp;rsquo;s local development setup, where it&amp;rsquo;s exactly right. Committed and run in CI, it silently short-circuits the federation. The pipeline either fails to find credentials, or finds the wrong ones.&lt;/p&gt;
&lt;p&gt;The rule is simple once you know it: the provider block that runs in CI must not name a &lt;code&gt;profile&lt;/code&gt;. Leave the chain free to find the web identity. It&amp;rsquo;s the kind of bug that teaches you to be precise about &lt;em&gt;which&lt;/em&gt; link of the credential chain you&amp;rsquo;re actually relying on.&lt;/p&gt;
&lt;h2 id="the-bottom-line"&gt;The bottom line
&lt;/h2&gt;&lt;p&gt;Giving CI an AWS access key means storing your most powerful, longest-lived credential in one of your most exposed systems. OIDC federation removes it entirely. The CI platform mints a short-lived signed token, AWS exchanges it via &lt;code&gt;AssumeRoleWithWebIdentity&lt;/code&gt; for hour-long credentials against a role whose trust policy names the exact pipeline, and nothing permanent is stored.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;terraform-aws-bootstrap&lt;/code&gt; builds the AWS side, the identity provider and the trust policy; the CI components build the consuming side, the token request. The one trap is a hardcoded &lt;code&gt;profile&lt;/code&gt; in the provider block, which short-circuits the SDK&amp;rsquo;s credential chain before it reaches the web-identity path. Get that right, and a pipeline deploys to AWS as a verifiable, short-lived identity, with no key to steal.&lt;/p&gt;</description></item><item><title>The chicken-and-egg of remote state</title><link>https://blog-570662.gitlab.io/the-chicken-and-egg-of-remote-state/</link><pubDate>Wed, 06 May 2026 00:00:00 +0000</pubDate><guid>https://blog-570662.gitlab.io/the-chicken-and-egg-of-remote-state/</guid><description>&lt;img src="https://blog-570662.gitlab.io/the-chicken-and-egg-of-remote-state/cover-the-chicken-and-egg-of-remote-state.png" alt="Featured image of post The chicken-and-egg of remote state" /&gt;&lt;p&gt;Here&amp;rsquo;s a puzzle that every infrastructure-as-code setup hits exactly once, right at the very beginning, and then never again. An OpenTofu stack stores its state in a backend. &lt;a class="link" href="https://blog-570662.gitlab.io/the-bootstrap-that-does-almost-nothing/" &gt;The bootstrap stack&lt;/a&gt; I wrote about last time has a particular job, and part of that job is to &lt;em&gt;create&lt;/em&gt; the backend that remote state lives in. So where does the bootstrap stack store its own state, on the very first run, before it&amp;rsquo;s built the place state is supposed to go?&lt;/p&gt;
&lt;h2 id="where-does-the-state-of-the-thing-that-makes-the-state-store-live"&gt;Where does the state of the thing that makes the state store live?
&lt;/h2&gt;&lt;p&gt;That&amp;rsquo;s the puzzle, and it&amp;rsquo;s a real ordering deadlock rather than a riddle.&lt;/p&gt;
&lt;p&gt;An OpenTofu stack keeps a state file, and for anything shared that state file lives in a remote backend: on AWS, an S3 bucket. Fine. But the bootstrap stack has a particular job, and part of that job is to &lt;em&gt;create the S3 bucket that remote state lives in&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;So walk through the first run. Bootstrap has never been applied. The state bucket doesn&amp;rsquo;t exist, because creating it is what bootstrap is for. Bootstrap needs somewhere to store its own state. The only place that would make sense is the bucket it&amp;rsquo;s about to create, which isn&amp;rsquo;t there yet. The thing that builds the state store can&amp;rsquo;t store its state in the state store.&lt;/p&gt;
&lt;h2 id="run-local-then-migrate"&gt;Run local, then migrate
&lt;/h2&gt;&lt;p&gt;The way out is a two-step that OpenTofu supports directly.&lt;/p&gt;
&lt;p&gt;Bootstrap starts configured with a &lt;em&gt;local&lt;/em&gt; backend: &lt;code&gt;backend &amp;quot;local&amp;quot; {}&lt;/code&gt;. State is just a file on the operator&amp;rsquo;s machine. With that in place, the first &lt;code&gt;tofu apply&lt;/code&gt; runs. It creates the S3 bucket and the KMS key, and records all of it in the local state file.&lt;/p&gt;
&lt;p&gt;Now the bucket exists. So the backend configuration is rewritten to point at it: an &lt;code&gt;s3&lt;/code&gt; backend block naming the new bucket. Then &lt;code&gt;tofu init -migrate-state&lt;/code&gt;. OpenTofu sees the backend has changed, picks up the local state file, and copies it into the S3 bucket. From that point on, bootstrap&amp;rsquo;s own state lives in the bucket that bootstrap created. The egg has laid the chicken.&lt;/p&gt;
&lt;p&gt;The local backend was a scaffold. It existed for exactly one apply, to break the ordering deadlock, and then the state moved off it and it was never used again.&lt;/p&gt;
&lt;h2 id="it-happened-twice"&gt;It happened twice
&lt;/h2&gt;&lt;p&gt;The &lt;code&gt;infra&lt;/code&gt; repo actually did this migration twice, and the second time is the proof that the pattern is general rather than a one-off trick.&lt;/p&gt;
&lt;p&gt;The first migration was the one above: local to S3, at the very start. The second came later, during the &lt;a class="link" href="https://blog-570662.gitlab.io/why-we-left-github-for-gitlab/" &gt;move from GitHub to GitLab&lt;/a&gt;. GitLab offers a managed HTTP state backend, and &lt;code&gt;infra&lt;/code&gt; chose to use it. So the backend block was rewritten again, this time from &lt;code&gt;s3&lt;/code&gt; to &lt;code&gt;http&lt;/code&gt;, and &lt;code&gt;tofu init -migrate-state&lt;/code&gt; ran again, copying the state from the S3 bucket to GitLab&amp;rsquo;s backend.&lt;/p&gt;
&lt;p&gt;The same move, twice, against three different backends. That&amp;rsquo;s the useful lesson hiding in the chicken-and-egg story. State is portable. The backend is just &lt;em&gt;where you currently keep it&lt;/em&gt;, not a property of the stack itself, and moving it is a routine, supported operation rather than surgery.&lt;/p&gt;
&lt;h2 id="why-this-is-the-honest-answer-not-a-hack"&gt;Why this is the honest answer, not a hack
&lt;/h2&gt;&lt;p&gt;It&amp;rsquo;s easy to look at &amp;ldquo;apply once with a local backend, then migrate&amp;rdquo; and feel it&amp;rsquo;s a bit of a smell, a workaround for something that should have been cleaner.&lt;/p&gt;
&lt;p&gt;It isn&amp;rsquo;t. It&amp;rsquo;s the honest answer to a real ordering problem, and the alternatives are worse.&lt;/p&gt;
&lt;p&gt;The obvious alternative is to create the state bucket by hand, in the console, before running bootstrap at all. But then the most important bucket in the account is unmanaged. It exists outside every OpenTofu graph, nobody&amp;rsquo;s code describes it, its encryption and policy and &lt;code&gt;prevent_destroy&lt;/code&gt; are whatever someone clicked that day, and it drifts. The local-then-migrate dance avoids exactly that. The bucket is created &lt;em&gt;by bootstrap&lt;/em&gt;, described in code, and tracked in bootstrap&amp;rsquo;s own state from its very first apply. It&amp;rsquo;s managed from birth.&lt;/p&gt;
&lt;p&gt;The chicken-and-egg isn&amp;rsquo;t a flaw to be embarrassed about. It&amp;rsquo;s just the shape of the problem when a stack has to build its own foundations, and OpenTofu&amp;rsquo;s &lt;code&gt;-migrate-state&lt;/code&gt; is the supported tool for exactly that shape.&lt;/p&gt;
&lt;h2 id="pulling-it-together"&gt;Pulling it together
&lt;/h2&gt;&lt;p&gt;Every OpenTofu stack needs a backend to store state, and the bootstrap stack&amp;rsquo;s job is to &lt;em&gt;create&lt;/em&gt; the backend, so on its first run the bucket it needs doesn&amp;rsquo;t yet exist.&lt;/p&gt;
&lt;p&gt;The resolution is to run bootstrap once with a local backend, let that apply create the bucket and key, then rewrite the backend configuration and &lt;code&gt;tofu init -migrate-state&lt;/code&gt; the state into the bucket bootstrap just made. The &lt;code&gt;infra&lt;/code&gt; repo did it twice, local to S3 and later S3 to GitLab, which shows the real point: state is portable, and the backend is just where you keep it. Doing it this way, rather than hand-creating the bucket, is what keeps that critical bucket managed in code from its very first day.&lt;/p&gt;</description></item><item><title>A state bucket that defends itself</title><link>https://blog-570662.gitlab.io/a-state-bucket-that-defends-itself/</link><pubDate>Sat, 02 May 2026 00:00:00 +0000</pubDate><guid>https://blog-570662.gitlab.io/a-state-bucket-that-defends-itself/</guid><description>&lt;img src="https://blog-570662.gitlab.io/a-state-bucket-that-defends-itself/cover-a-state-bucket-that-defends-itself.png" alt="Featured image of post A state bucket that defends itself" /&gt;&lt;p&gt;OpenTofu&amp;rsquo;s remote state file is, quietly, the most sensitive thing in an infrastructure repo. It&amp;rsquo;s a plain JSON document listing every resource you manage, every ID, and, depending on your providers, the odd secret in clear text. So the S3 bucket that holds it can&amp;rsquo;t just be a bucket. It has to actively defend itself, on three separate fronts.&lt;/p&gt;
&lt;h2 id="the-most-sensitive-file-in-the-repo"&gt;The most sensitive file in the repo
&lt;/h2&gt;&lt;p&gt;OpenTofu, like Terraform, keeps a state file: a JSON document recording every resource the stack manages, its real-world ID, and its attributes. It&amp;rsquo;s how the tool knows what already exists. It&amp;rsquo;s also, quietly, the most sensitive file in the whole repo. It can hold resource identifiers an attacker would value, and depending on the providers in play it can hold secret values in clear text.&lt;/p&gt;
&lt;p&gt;Three bad things can happen to it. It can be deleted, and now the tool has forgotten everything it manages. It can be read by someone who shouldn&amp;rsquo;t. It can be corrupted by two runs writing at once. The bucket that holds remote state has to defend against all three, and &lt;code&gt;terraform-aws-bootstrap&lt;/code&gt;&amp;rsquo;s &lt;code&gt;state-backend&lt;/code&gt; module is built around doing exactly that.&lt;/p&gt;
&lt;h2 id="the-dynamodb-lock-table-is-gone"&gt;The DynamoDB lock table is gone
&lt;/h2&gt;&lt;p&gt;Start with the corruption problem, because the answer changed recently.&lt;/p&gt;
&lt;p&gt;The long-standing pattern for remote state on AWS was an S3 bucket &lt;em&gt;plus&lt;/em&gt; a DynamoDB table. S3 held the state; the DynamoDB table held a lock, so two &lt;code&gt;apply&lt;/code&gt; runs couldn&amp;rsquo;t write at once. Everyone who&amp;rsquo;s done Terraform on AWS has provisioned that table, probably more times than they&amp;rsquo;d care to count.&lt;/p&gt;
&lt;p&gt;OpenTofu 1.10 made it unnecessary. The S3 backend gained &lt;code&gt;use_lockfile&lt;/code&gt;, which does the locking with a small lock &lt;em&gt;object&lt;/em&gt; in the same bucket, using S3&amp;rsquo;s conditional-write support. No separate table. The state backend is now genuinely one bucket and one key, with the lock living beside the state. It&amp;rsquo;s one fewer resource to create, one fewer thing to pay for, and one fewer moving part to reason about. The module takes the new path, and the DynamoDB table simply isn&amp;rsquo;t there.&lt;/p&gt;
&lt;h2 id="a-bucket-you-cant-delete-by-accident"&gt;A bucket you can&amp;rsquo;t delete by accident
&lt;/h2&gt;&lt;p&gt;Deletion is guarded with &lt;code&gt;lifecycle { prevent_destroy = true }&lt;/code&gt; on the bucket. With that set, OpenTofu refuses to produce a plan that would destroy the bucket. A stray &lt;code&gt;tofu destroy&lt;/code&gt;, a refactor that drops the resource, an accidental rename: all of them fail loudly instead of quietly taking the state bucket with them.&lt;/p&gt;
&lt;p&gt;This is also why the &lt;code&gt;state-backend&lt;/code&gt; module is hand-rolled from raw &lt;code&gt;aws_s3_bucket&lt;/code&gt; resources rather than wrapping a community module like &lt;code&gt;terraform-aws-modules/s3-bucket&lt;/code&gt;. &lt;code&gt;prevent_destroy&lt;/code&gt; has to sit on the actual resource, and a &lt;code&gt;lifecycle&lt;/code&gt; block isn&amp;rsquo;t something you can pass into a wrapper module as an input. Hand-rolling the bucket keeps &lt;code&gt;prevent_destroy&lt;/code&gt; somewhere you can put it and, just as importantly, somewhere the next reader can see it. (There&amp;rsquo;s a whole post coming on &lt;a class="link" href="https://blog-570662.gitlab.io/why-i-hand-rolled-every-module/" &gt;why I hand-rolled every module&lt;/a&gt;; this is one of the reasons in miniature.)&lt;/p&gt;
&lt;h2 id="reject-anything-encrypted-wrong"&gt;Reject anything encrypted wrong
&lt;/h2&gt;&lt;p&gt;Confidentiality is the subtle one, because the obvious control isn&amp;rsquo;t enough.&lt;/p&gt;
&lt;p&gt;The bucket has a default encryption configuration: server-side encryption with the customer-managed KMS key. But default encryption is a &lt;em&gt;default&lt;/em&gt;. A client making a &lt;code&gt;PutObject&lt;/code&gt; call can override it per request, asking for plain &lt;code&gt;AES256&lt;/code&gt; or a different KMS key, and S3 will honour the override.&lt;/p&gt;
&lt;p&gt;So the module doesn&amp;rsquo;t rely on the default. The bucket policy explicitly denies the upload it doesn&amp;rsquo;t want. It denies any request not over TLS. It denies any &lt;code&gt;PutObject&lt;/code&gt; that isn&amp;rsquo;t using SSE-KMS. And it denies any &lt;code&gt;PutObject&lt;/code&gt; that names the &lt;em&gt;wrong&lt;/em&gt; KMS key. The default encryption config says &amp;ldquo;this is what you get if you don&amp;rsquo;t ask&amp;rdquo;; the bucket policy says &amp;ldquo;and you&amp;rsquo;re not allowed to ask for anything else&amp;rdquo;. State can only ever land encrypted, in transit and at rest, under the one key the module controls.&lt;/p&gt;
&lt;p&gt;One small companion setting: &lt;code&gt;bucket_key_enabled&lt;/code&gt;. With per-object SSE-KMS, every object operation is also a KMS API call, which costs money and can throttle. An S3 Bucket Key collapses those into far fewer KMS calls, cutting per-object KMS traffic by well over ninety per cent. It&amp;rsquo;s a one-line setting the module turns on and most people forget exists.&lt;/p&gt;
&lt;h2 id="in-short"&gt;In short
&lt;/h2&gt;&lt;p&gt;Remote state is the most sensitive file an infrastructure repo has, and the bucket that holds it has to defend against deletion, disclosure and corruption.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;terraform-aws-bootstrap&lt;/code&gt;&amp;rsquo;s state backend handles corruption with OpenTofu 1.10&amp;rsquo;s &lt;code&gt;use_lockfile&lt;/code&gt;, dropping the old DynamoDB lock table entirely. It guards deletion with &lt;code&gt;prevent_destroy&lt;/code&gt;, which is also why the bucket is hand-rolled rather than wrapped. And it guards confidentiality with a bucket policy that denies non-TLS traffic and denies any upload not encrypted with the right KMS key, because default encryption is only a default and a client can override it. The state bucket isn&amp;rsquo;t just a place to put state. It&amp;rsquo;s built to refuse every wrong thing that could happen to it.&lt;/p&gt;</description></item><item><title>The bootstrap that does almost nothing</title><link>https://blog-570662.gitlab.io/the-bootstrap-that-does-almost-nothing/</link><pubDate>Fri, 01 May 2026 00:00:00 +0000</pubDate><guid>https://blog-570662.gitlab.io/the-bootstrap-that-does-almost-nothing/</guid><description>&lt;img src="https://blog-570662.gitlab.io/the-bootstrap-that-does-almost-nothing/cover-the-bootstrap-that-does-almost-nothing.png" alt="Featured image of post The bootstrap that does almost nothing" /&gt;&lt;p&gt;A brand-new AWS account is a slightly nerve-wracking thing. It can do almost anything, it&amp;rsquo;s hardened against almost nothing, and the list of stuff you ought to set up before you trust it with anything real is long. The natural instinct is to write one big &amp;ldquo;set up the account&amp;rdquo; module that does the whole list in a single apply. I want to talk you out of that, because the bootstrap module I&amp;rsquo;m happiest with does almost nothing, on purpose.&lt;/p&gt;
&lt;h2 id="the-first-apply-problem"&gt;The first-apply problem
&lt;/h2&gt;&lt;p&gt;A brand-new AWS account is not ready for anything serious. Before you&amp;rsquo;d responsibly run real infrastructure into it, you want an account baseline: a password policy, account-wide S3 public-access blocking, default EBS encryption, CloudTrail, AWS Config, GuardDuty, alerting, a sensible human operator role. It&amp;rsquo;s a long list, and all of it matters.&lt;/p&gt;
&lt;p&gt;The instinct, faced with that list, is to write one big &amp;ldquo;set up the account&amp;rdquo; module and have it do everything. One &lt;code&gt;tofu apply&lt;/code&gt;, a fully prepared account, done.&lt;/p&gt;
&lt;p&gt;That instinct is worth resisting, and &lt;code&gt;terraform-aws-bootstrap&lt;/code&gt; resists it deliberately.&lt;/p&gt;
&lt;h2 id="three-things-and-a-hard-line"&gt;Three things, and a hard line
&lt;/h2&gt;&lt;p&gt;&lt;code&gt;terraform-aws-bootstrap&lt;/code&gt; does three things:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;state-backend&lt;/code&gt;&lt;/strong&gt;, an S3 bucket and a customer-managed KMS key to hold remote Terraform state.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;automation-iam&lt;/code&gt;&lt;/strong&gt;, an OIDC identity provider and an IAM role that CI assumes to apply everything else.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;nuke-config&lt;/code&gt;&lt;/strong&gt;, which renders an &lt;a class="link" href="https://github.com/ekristen/aws-nuke" target="_blank" rel="noopener"
 &gt;aws-nuke&lt;/a&gt; configuration scoped to the account, for tearing a throwaway account back down.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;That&amp;rsquo;s the whole module. Account hardening, CloudTrail, AWS Config, GuardDuty, the operator role, the alerting: none of it is in here. And it&amp;rsquo;s not absent by accident. The README has a section headed &amp;ldquo;what&amp;rsquo;s deliberately NOT in scope&amp;rdquo; that lists those exclusions out loud. The boundary is written down, because the boundary is the design.&lt;/p&gt;
&lt;h2 id="why-the-line-is-exactly-there"&gt;Why the line is exactly there
&lt;/h2&gt;&lt;p&gt;The reason the line sits where it does is the most useful idea in the module.&lt;/p&gt;
&lt;p&gt;Everything bootstrap excludes belongs in a &lt;em&gt;separate&lt;/em&gt; stack, applied &lt;em&gt;through the automation role bootstrap creates&lt;/em&gt;. Bootstrap&amp;rsquo;s only job is to get the account to the point where the next &lt;code&gt;tofu apply&lt;/code&gt; can run properly: somewhere to store state, and an identity to run as. Once those two things exist, hardening the account isn&amp;rsquo;t a special bootstrapping act. It&amp;rsquo;s just another apply, done the normal way: in CI, reviewed, versioned, deployed through the role.&lt;/p&gt;
&lt;p&gt;So the account baseline doesn&amp;rsquo;t need to be bundled into the bootstrap. It needs to be &lt;em&gt;downstream&lt;/em&gt; of it. Bootstrap builds the on-ramp; it doesn&amp;rsquo;t also have to be the motorway.&lt;/p&gt;
&lt;h2 id="a-narrow-module-stays-re-runnable"&gt;A narrow module stays re-runnable
&lt;/h2&gt;&lt;p&gt;There&amp;rsquo;s a practical payoff to the narrowness, and it&amp;rsquo;s about fear.&lt;/p&gt;
&lt;p&gt;Bootstrap is the one stack that &lt;em&gt;can&amp;rsquo;t&lt;/em&gt; be applied through CI, because it&amp;rsquo;s what creates the CI identity in the first place. It runs locally, by a human, rarely. That&amp;rsquo;s exactly the kind of operation you want to be small, boring, and safe to repeat.&lt;/p&gt;
&lt;p&gt;A bootstrap module that also did account hardening would be a large, stateful thing managing dozens of resources. Re-running it would be a held-breath operation. Keeping it to three concerns keeps it the opposite: a small stack you can read top to bottom, re-run without anxiety, and reason about completely. The narrowness isn&amp;rsquo;t minimalism for its own sake. It&amp;rsquo;s what keeps the one human-applied stack trustworthy.&lt;/p&gt;
&lt;h2 id="the-boundary-is-the-feature"&gt;The boundary is the feature
&lt;/h2&gt;&lt;p&gt;It&amp;rsquo;s tempting to judge a module by how much it does. A bootstrap module is the case where that&amp;rsquo;s exactly backwards. Its value is in how cleanly it stops.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;terraform-aws-bootstrap&lt;/code&gt; does the bare minimum to make an account ready for the next apply, writes down everything it refuses to do, and hands off to a downstream stack for all of it. The next post follows the trickiest of its three jobs: the state backend has a &lt;a class="link" href="https://blog-570662.gitlab.io/the-chicken-and-egg-of-remote-state/" &gt;genuine chicken-and-egg problem&lt;/a&gt;, because it has to store Terraform state in a bucket Terraform hasn&amp;rsquo;t created yet.&lt;/p&gt;
&lt;h2 id="where-this-leaves-us"&gt;Where this leaves us
&lt;/h2&gt;&lt;p&gt;A fresh AWS account needs a long list of things before it&amp;rsquo;s safe, and the obvious move is one big module that does the lot. &lt;code&gt;terraform-aws-bootstrap&lt;/code&gt; deliberately does only three: a state backend, a CI identity, and an account-scrub config. Everything else is written down as out of scope.&lt;/p&gt;
&lt;p&gt;The boundary is the design. The excluded work belongs in a downstream stack applied through the CI role bootstrap creates, so hardening is just a normal reviewed apply rather than a bootstrapping special case. And keeping the one human-run, locally-applied stack small is what keeps it safe to re-run. A bootstrap module is judged by where it stops.&lt;/p&gt;</description></item></channel></rss>