Why a Single KMS Key on a Multi-Tenant S3 Bucket is a Blast Radius Waiting to Happen

In The Blast Radius I keep coming back to the same question: when something goes wrong, how far does the damage spread? Not if something goes wrong. When. Because in my experience the mistake that hurts you is rarely a zero-day; it is a tired engineer approving a pull request that changes arn:aws:s3:::customer-data/tenant-42/* to arn:aws:s3:::customer-data/* at five o’clock on a Friday.

TL;DR:

• SSE-KMS gives you two authorisation gates: S3 decides if you may fetch the ciphertext, KMS decides if you may decrypt it. They only protect you if they can fail independently.
• One KMS key for every tenant means every tenant’s role already holds kms:Decrypt. An S3 wildcard then exposes all of them; the second gate contributes nothing.
• Per-object data encryption keys do not isolate tenants. Every DEK is wrapped by the same key and S3 unwraps them transparently. Envelope encryption changes how keys are stored, not who may use them.
• Fix: one customer managed key per tenant (or per tier), key policies scoped to named principals rather than account root, encryption context conditions, ABAC on key and role tags, VPC endpoint conditions, and alerting on cross-tenant Decrypt calls.

This post is about one specific, very common pattern: a single S3 bucket holding data for many tenants, encrypted with a single AWS Key Management Service (KMS) key. I should say up front that storing multiple tenants’ data in a single bucket is not a recommended practice in itself; separate buckets, or at least separate accounts for your most sensitive customers, is where you want to be. But plenty of systems are built this way for cost or convenience, and if you are going to do it, the key strategy decides how bad a mistake gets. At first glance, the single-key version looks fine. The data is encrypted at rest, you have a nice tick in the compliance checklist, and the IAM policies scope each tenant to their own prefix. What could go wrong?

More than enough. Let’s dig deeper into why.

The Analogy

Imagine an apartment building with fifty flats. Every flat has its own lock and every resident has their own key. Your key opens your door and nobody else’s. So far, so good.

The building manager, of course, has a passe-partout: one key that opens every flat, because sometimes he needs to get in with someone for maintenance. He keeps it in his pocket and one day he accidentally drops it in the lobby, and the resident of flat 12 picks it up.

Nothing about flat 12’s own key has changed. The locks are all still good locks. But one mistake by one person, and one resident now has access to fifty homes, because the key he found opens all of them. The blast radius of a dropped key is the entire building.

Now imagine instead that the manager has one master key per tenant. Each one opens everything that belongs to that tenant: their flat, their garage, their storage unit in the basement. If he drops one of those, the resident who finds it can get into one tenant’s flat, garage and storage unit. Still a problem, still a breach, and still everything that tenant owns, but one tenant’s property rather than fifty.

That is exactly the difference between a single KMS key on a multi-tenant bucket and a key per tenant. The dropped key is the IAM mistake. The passe-partout is the shared KMS key. And all fifty locks are worthless if there’s one key that, in the wrong hands, opens all of them.

How S3 and KMS Are Supposed to Protect You

To understand the problem, it helps to be precise about what happens when someone reads an object encrypted with server-side encryption using KMS (SSE-KMS).

To successfully GetObject, the caller needs two independent permissions:

1. S3 authorisation. The caller’s IAM identity policy, the bucket policy (and any access point policy) must allow s3:GetObject on that object ARN.

2. KMS authorisation. The caller must also be allowed kms:Decrypt on the key that encrypted the object. That permission is evaluated against the KMS key policy plus the caller’s IAM policy.

Strictly there is a third gate in front of both of these: network access. A bucket policy can refuse any request that does not arrive through a specific VPC endpoint (aws:SourceVpce) or from an allowed address range, and Block Public Access stops the bucket ever being reachable from the internet at all. That is a real control and you should use it. But it is a control on where the request comes from, not who is asking, and in a multi-tenant system every tenant’s workload legitimately lives inside the same VPC and hits the same endpoint. So for the cross-tenant case it is the S3 and KMS gates that have to do the work, and those are what this post is about.

This is known as defence in depth, and it is the entire reason KMS is worth paying for. S3 decides whether you may fetch the ciphertext; KMS decides whether you may turn it back into plaintext. A mistake in one layer should be caught by the other. If someone accidentally makes an S3 prefix readable, the data is still gibberish unless the reader also has kms:Decrypt on the right key.

But here’s where my concern arises. That second gate only protects you if the second gate is actually different for different data.

Why One Key Collapses the Second Gate

With a single key for the whole bucket, every legitimate tenant workload needs kms:Decrypt on that one key. Tenant A’s service role has it. Tenant B’s service role has it. So does the batch job, the reporting Lambda, the support tooling, and probably the CI pipeline that ran the migration last year and nobody cleaned up.

Now consider the Friday afternoon IAM mistake. Someone widens an S3 permission from a tenant-specific prefix to the whole bucket. Perhaps it is a wildcard in a Resource element, perhaps a Condition on s3:prefix that got deleted during a refactor, perhaps a bucket policy Principal that was meant to be one role and ended up as the account root.

Whatever the cause, the S3 gate is now open for every tenant’s data. And the KMS gate? It was already open, because the principal in question legitimately had kms:Decrypt on the shared key. Your second layer of defence contributed precisely nothing.

The blast radius of one IAM typo went from “one tenant” to “all tenants”. And what you paid for using a KMS gave you precisely nothing extra.

“But Every Object Has Its Own Key!”

This is the objection I have heard more than once, so let’s put that myth to bed once and for all.

SSE-KMS uses envelope encryption. S3 does not encrypt your objects directly with the KMS key. Instead, for each object it asks KMS to generate a fresh data encryption key (DEK), encrypts the object with that DEK, and stores the DEK alongside the object wrapped (encrypted) by your KMS key. On read, S3 sends the wrapped DEK back to KMS, KMS unwraps it, and S3 uses the plaintext DEK to decrypt the object. So yes, technically Tenant A’s objects and Tenant B’s objects are encrypted with different keys. Thousands of them.

At first glance that sounds like isolation. It isn’t, for two reasons.

First, every one of those DEKs is wrapped by the same KMS key. The DEK is not an authorisation boundary; it is an implementation detail of how KMS keeps the master key material inside the HSM. The question KMS asks on every read is not “is this principal allowed this DEK?” but “is this principal allowed kms:Decrypt on the wrapping key?” With one wrapping key for the whole bucket, the answer is the same for every object in it. A million DEKs, one gate.

Second, and more fundamentally, SSE-KMS is still transparent encryption. The decrypt happens inside S3, on your behalf, before the bytes ever leave AWS. The caller never sees a DEK, never sees ciphertext, never has to do anything cryptographic at all. If S3 and KMS both say “yes”, plaintext comes out of GetObject exactly as if the object had never been encrypted. From the perspective of an over-privileged IAM principal, the encryption is invisible. That is what “transparent” means, and it is a feature, right until you start believing it gives you tenant isolation.

This is the bit I want to get through to people: envelope encryption changes how the keys are stored, not who is allowed to use them. Per-object DEKs protect you against someone walking off with the physical disks, and they limit the value of any single key: because no DEK is ever reused, a key that leaks or is recovered through an implementation flaw decrypts one object and nothing else, and no key ever encrypts enough data to become a worthwhile cryptanalysis target. They do nothing whatsoever against someone who has been granted the wrong IAM permissions, because that person is not attacking the cryptography. They are simply asking S3 nicely, and S3 is saying yes.

If you want the DEK layer to mean something at the tenant level, you need either a different wrapping key per tenant (so the kms:Decrypt check differs), or client-side encryption where the application holds the keys and S3 only ever sees ciphertext. Transparent server-side encryption with one wrapping key gives you neither.

It Gets Worse With Overly Broad KMS Grants

The pattern I see most often in the wild makes this even more uncomfortable. Because managing per-role KMS permissions is fiddly, teams write key policies like this:

{
  “Sid”: “AllowUseOfTheKey”,
  “Effect”: “Allow”,
  “Principal”: { “AWS”: “arn:aws:iam::123456789012:root” },
  “Action”: [“kms:Decrypt”, “kms:GenerateDataKey*”],
  “Resource”: “*”
}

That delegates the decision entirely to IAM within the account. The key policy is no longer an independent control at all. Now the S3 gate and the KMS gate are evaluated from the same set of IAM policies, so one mistake in IAM opens both simultaneously. You have exactly one lock and you’ve told yourself there are two.

What “Blast Radius” Actually Means Here

Let me be very clear about what is in scope when this goes wrong, because “all tenants” is abstract until you write it down:

• Confidentiality across tenant boundaries. Tenant A’s engineers, or a compromised Tenant A workload, can read Tenant B’s objects. In a regulated context that is a reportable breach, not an incident.

• Every tenant, not the worst tenant. Your risk is no longer bounded by your largest or most sensitive customer. It is the union of all of them.

• Time. Overly permissive IAM often sits unnoticed for months. CloudTrail will faithfully record every GetObject and every Decrypt, but if you are not alerting on cross-tenant access patterns you will find out from a customer, not from a dashboard.

• Revocation cost. With one key, you cannot revoke a single tenant’s ability to decrypt without breaking everyone. You cannot rotate or disable the key for one tenant. Crypto-shredding one tenant’s data on offboarding is impossible.

• Compromised credentials. Everything above applies equally when the “mistake” is an attacker holding stolen keys for a role that was scoped correctly but sits in an account where KMS is not doing its job.

Questions to Ask About Your Own Setup

If you are not sure whether this applies to you, these are the questions I would want answered:

• How many distinct KMS keys encrypt tenant data in this bucket? If the answer is “one”, keep reading.

• Does the key policy delegate to the account root, or does it name specific principals?

• If I gave a random service role s3:GetObject on the whole bucket right now, would it be able to decrypt everything?

• Can you offboard a tenant by disabling or scheduling deletion of their key?

• Would a CloudTrail query show you a principal decrypting data outside its own tenant?

If those questions make you uncomfortable, good. That is the point.

Reducing the Blast Radius

The fix is not exotic. The principle is simple: make the KMS gate genuinely different per tenant, so that one IAM mistake cannot open both gates for everyone.

1. One KMS key per tenant

Create a customer managed key per tenant and encrypt each tenant’s objects with their own key. Now the S3 mistake still exposes ciphertext across tenants, but the KMS gate holds, because Tenant A’s role has never been granted kms:Decrypt on Tenant B’s key.

This also gives you clean offboarding: disable the key and Tenant B’s data is cryptographically gone, wherever copies of it may have ended up.

Note on cost and scale: each customer managed key costs a dollar a month plus API calls, and the default quota is 100,000 customer managed keys per region, adjustable through Service Quotas. For most SaaS products this is entirely manageable. If you have a very high tenant count you can group tenants into key pools by tier or region, which is still vastly better than one key for everyone. Enable S3 Bucket Keys to cut the KMS request volume. S3 only shares a bucket key between objects encrypted under the same KMS key, so with per-tenant keys you get one bucket key per tenant and the saving still applies. Usefully, S3 also fetches a fresh bucket key at least once per distinct requester, precisely so that each role’s access to each key still lands in CloudTrail.

2. Lock the key policy to specific principals

Do not delegate the whole key policy to the account root. Name the roles that may use each key. Yes, it is more to manage. That is precisely the friction that stops the second gate from silently becoming a copy of the first.

3. Use encryption context conditions

S3 always passes an aws:s3:arn encryption context when it calls KMS: the object ARN by default, or the bucket ARN when S3 Bucket Keys are enabled. You can add a kms:EncryptionContext:aws:s3:arn condition to each tenant’s key policy so the key can only ever be used for that tenant’s bucket (or, without Bucket Keys, that tenant’s prefix). One trap to know about: if you condition on an object or prefix ARN and then switch Bucket Keys on, the context becomes the bucket ARN, the condition stops matching and every request fails. Pick one and write the policy for it. If you use client-side encryption you can go further and include a tenant ID in the encryption context and condition on that. Either way, it means a key cannot be repurposed by accident.

4. Tag-based access control for scale

If per-role key policies do not scale for you, tag both the keys and the roles with a tenant identifier and use attribute-based access control (ABAC) conditions such as aws:ResourceTag/tenant matching aws:PrincipalTag/tenant. This keeps the KMS gate independent of the S3 gate without a policy explosion. Just remember that whoever can change tags can change access, so guard tagging permissions accordingly.

5. Consider separating the S3 side too

Per-tenant S3 Access Points, or per-tenant buckets for your most sensitive customers, shrink the S3 gate as well. Both gates smaller is better than one gate smaller. But if I could only change one thing, I would change the key strategy first, because that is the gate that is supposed to catch the IAM mistake.

6. Keep the network gate closed too

Enable Block Public Access at the account level, and add a Deny in the bucket policy for any request not arriving via your VPC endpoint. This does not help with the cross-tenant case above, but it does mean the IAM mistake has to be exploited from inside your network rather than from anywhere on earth, which is a smaller pool of attackers and a much better audit trail.

7. Detect what you can’t prevent

Alert on Decrypt calls where the calling principal’s tenant does not match the key’s tenant tag, and on IAM changes that widen S3 resource scope. Note that with Bucket Keys enabled you will see fewer KMS events in CloudTrail and they will carry the bucket ARN rather than the object ARN, so build the detection on principal versus key, not on object paths. Preventive controls fail. Detective controls tell you how long they have been failing.

Note to architects: if your threat model has a box labelled “encryption at rest” and you have ticked it because SSE-KMS is enabled, go back and ask what that control actually does when IAM is wrong. If the honest answer is “nothing”, the box should not be ticked.

Note to engineers reviewing IAM changes: any pull request that touches an S3 Resource or Condition element on a multi-tenant bucket deserves the same scrutiny you would give a change to your authentication code. It is authorisation code.

Final Thoughts

There is an Italian saying, chi ha una chiave sola, apre una porta sola (whoever has one key opens one door). The design I’ve described inverts it: whoever has the one key opens every door, and the only thing standing between them and your customers’ data is a Friday afternoon IAM change being wrong.

There are three things standing between a request and a tenant’s plaintext: the authorisation to reach the bucket (network), the authorisation to read the object (S3), and the authorisation to decrypt it (KMS). Each one is only a control if it can say no when the others have said yes. Encryption at rest earns its place in that list only when the authorisation to decrypt is meaningfully separate from the authorisation to read. One key for a multi-tenant bucket throws that separation away and leaves you with a single point of failure that happens to be wrapped in AES-256. I just want to re-iterate at this point separate S3 buckets is still preferable for improved separation.

Per-tenant keys are not free and not glamorous. But when the inevitable mistake happens, they are the difference between “we exposed one customer” and “we exposed all of them”. That is what blast radius is about.

One last thing, because it is the one that actually determines whether any of this gets fixed. The key strategy for a bucket is a decision that costs nothing to change on a whiteboard and a great deal to change once the bucket is full, the IAM is written, the application is wired to one key ARN and the release date is set. Threat modelling at design time finds it while it is still a whiteboard decision. Threat modelling as a tick box exercise at the end of the project finds exactly the same problem, but now the fix is a migration, the schedule slips, and the threat model gets blamed for a delay that was really caused by not asking the question when it was free to answer. The question is the same either way. Only the price changes.

Until then, go and count your keys. It’s almost as much fun as counting sheep.