Secrets, keys and service accounts

Everything built so far answers one question: given a session holding role R, what may it see? Row policies filter, masking policies transform, grants decide which objects exist at all. All of it is correct, and all of it takes the role on trust. The warehouse has no way of knowing that the session claiming to be the nightly dbt run is actually a laptop in a coffee shop with a copied password.

That is why credentials belong on the same scoreboard as policies. A leaked key for a service account that holds fr_finance is fr_finance, for anybody. A leaked key for the account that owns the vault schema is every surrogate in the modelling view, walked back to a name. The controls did not fail; they were never consulted, because the session presented a valid identity.

• Credential - anything a session presents to prove it is who it says: a password, an API token, a private key, a certificate.
• Secret - any string whose disclosure grants access. Credentials are secrets; so is the hashing key from session 5, and so is an encryption key.
• Service account - an identity used by software rather than a person. A pipeline, an orchestrator, a BI tool's connection, a model training job.
• Secrets manager - a system whose only job is to store secrets encrypted, hand them to authorised callers at runtime, log every read and make rotation possible.
• Rotation - replacing a credential with a new one and retiring the old one, on a schedule and on demand.
• Short-lived credential - one that expires on its own after minutes or hours, issued on request against a longer-lived identity. Nothing to rotate, because nothing lives long enough to steal usefully.
• Blast radius - everything a compromised credential can reach. The thing one-account-per-pipeline is designed to shrink.

The professional rule that closes all four is one sentence. Credentials live in a secrets manager and are injected into the process at runtime. Not in a repo, not in a profile file, not in a notebook, not in a chat message. The reason is not tidiness: each of the four places has a wider audience and a longer memory than the vault, and a secret is only as secret as the widest place it has ever been.

Human accounts have accumulated protections over twenty years of being attacked. Service accounts have almost none of them, and they usually hold more.

• No MFA. A pipeline cannot approve a push notification, so the credential is the whole proof of identity. Whoever holds the string is the account.
• No offboarding. When a person leaves, their access is revoked. When the engineer who created svc_dbt_nightly leaves, the account and its password stay exactly as they were, and everybody who ever saw that password keeps it.
• Shared across pipelines. One account created for the nightly build gets reused by the orchestrator, then by a model training job, because it already works. It now needs the union of three roles, and revoking it breaks three things, which is why it never gets revoked.
• Broadest grants in the estate. A pipeline that writes every table needs a role that can write every table. That role, held by an identity with no MFA and no leaver process, is the single most valuable string in the warehouse.

The modelling view is pseudonymised by a keyed hash. The key does not log anybody in, so it is easy to treat it as configuration rather than as a secret. It is the opposite: whoever can read it can regenerate every surrogate and join the modelling view straight back to names, emails and phone numbers. That is a re-identification of every customer in one query.

So it follows every rule on this page. It lives in the secrets manager, it is read at runtime by the one function that needs it, it has an owner, and it has a rotation schedule with a written plan for re-issuing surrogates when it turns. Rotating it is more disruptive than rotating a password, because every consumer of the modelling view has to re-key on the same day, which is exactly why the plan has to exist before the key leaks rather than after.

Any of the mainstream secrets managers will do. What matters is the set of properties, because a shared spreadsheet with a lock icon has none of them.

• Encrypted at rest, decrypted only for an authorised caller. The service reads its own secret; the engineer who deployed it does not need to see the value and usually should not.
• Every read is logged. Which identity fetched which secret, when. This is the audit trail that a password in a file can never have, and it is what tells you the blast radius after an incident.
• Versioned. Two versions of a secret can be valid at once, which is the property the whole rotation sequence depends on.
• Reachable from where the pipeline runs. If the runner cannot reach the manager, the engineer will paste the value into an environment variable "temporarily", and the four places are back.

What it must not be: a password manager designed for humans, shared through a team vault, with the warehouse password sitting next to somebody's airline login. Those tools are excellent for people and have no runtime injection, no per-service identity and no useful read log for a machine.

"We rotate credentials regularly" appears in every security questionnaire and is true in very few estates, because rotation without an owner and a date is a wish. The working version has three parts.

• A schedule per account. Ninety days is a common default for a long-lived password. The exact number matters less than the fact that it exists and that the next date is written down somewhere a review can read.
• A named owner per account. Not a team. A person whose name is next to svc_dbt_nightly and who is the one asked when it is overdue. Ownership is what turns a schedule into rotations that actually happen.
• A rotation on demand. Whenever the value has been seen by somebody who should not have it: a leaver, a log, a chat paste. The schedule is the floor, not the ceiling.

Rotation has one precondition that stops most first attempts. Nothing may depend on the old value being static. A hard-coded password in a notebook, a connection string baked into a container image, a second pipeline nobody knew was using the account: every one of those breaks the moment the old value dies. The rotation itself is five minutes. Finding those dependencies is the work, and the inventory in the build-along is where it starts.

Rotation shortens the life of a leak. The stronger move is to make sure there is no long-lived secret to leak in the first place. Two patterns do that, and both are the direction of travel across warehouse platforms.

• Key-pair authentication. The service holds a private key and the warehouse holds the matching public key. Nothing that grants access is ever sent over the wire or written into a profile, and the public half can be published without harm. Most platforms also let two public keys be registered at once, which makes rotation the five-step strip above with no shared-secret handling at all.
• Federated, short-lived tokens. The pipeline authenticates to the identity provider that already knows what it is, receives a token valid for an hour, and uses that. There is no password to rotate because there is no password. A stolen token is worthless by lunchtime.

Which of these a given warehouse supports, and with what syntax, is a platform question and changes often enough that the vendor's current documentation is the only reliable source. The design rule does not change: prefer the credential that expires on its own, and where a long-lived one is unavoidable, treat it as the liability it is.

Least privilege for people was sessions 1 and 2: one functional role per job, holding exactly the access roles that job needs. Machines get the same treatment, and for the same reason.

• One service account per pipeline. svc_dbt_nightly, svc_orchestrator, svc_churn_model, svc_bi_refresh. Not one svc_data that everything uses.
• Each with its own functional role. fr_pipeline_dbt can write the models it builds. fr_pipeline_churn can read the modelling view and nothing else. Neither can read the vault schema where the hashing key lives, because neither job needs to.
• The payoff is containment. A compromised orchestrator key cannot read the hashing secret, so the modelling view stays pseudonymous through the incident. A revoked key breaks exactly one thing, so revoking it is a decision somebody will actually take at 22:00 instead of deferring until Monday.

Nothing in tonight's session changes a number on the session 8 bench, and that is the point. The bench evaluates 150 requests against the policies for the role each request claims. It assumes the claim is true.

Read the last row as a credential story rather than a grant story. It is identical to the opening state, with every policy still defined and deployed, because a role above the policy layer is a role the policy layer does not reach. A stolen credential for an account holding that role produces exactly the same picture: 109 leaks, 30 standing privileges, and a warehouse that will tell an auditor every control is in place. A perfect policy set with a leaked service-account key is 109 leaks by another route. The bench cannot see the difference, and neither can the access review, which is why service accounts get their own row in session 7.

The order matters, and the wrong order is the intuitive one.

• Rotate before you investigate. The leaked value must stop working before anybody spends an hour working out who saw it. Follow the five-step strip: the pipeline stays up and the old value dies.
• Then read the logs for the old value's lifetime. The warehouse login history for that account, from the earliest date the value could have been exposed. Unfamiliar client, unfamiliar network, unfamiliar hour: any of those turns a leak into an incident.
• Assume the role's whole reach was read. Whatever the account's role could see in clear is what leaked. This is where one-account-per-pipeline pays off, because the answer is one pipeline's tables rather than everything.
• If it was the hashing key, the modelling view is re-identifiable. Rotate the key, re-issue surrogates to every consumer, and treat every past export of the view as containing real customer_ids.
• Write the finding into the schedule. The account that leaked gets a shorter rotation period and a review of how the value reached the place it was found.