Amazon S3 explained

The short answer

Amazon S3 (Simple Storage Service) stores objects, which are files plus a little metadata, in containers called buckets. You put and get whole objects over HTTPS by name. There are no folders, disks or file locks: just keys and the bytes stored under them.

Everything in a new bucket is private. Block Public Access is on, ACLs are off, and every new object is encrypted with S3-managed keys. Most S3 trouble comes from changing one of those defaults without understanding the others.

An analogy, and where it breaks

Think of a vast warehouse of labelled boxes. A bucket is your locked section of it. Each object is a sealed box, and its key is the label, such as photos/2026/beach.jpg. The section has a rule sheet on the door saying who may take which boxes (the bucket policy), and the building has a master switch that overrides any rule sheet saying “anyone” (Block Public Access).

Where it breaks:

• The slashes in a label are not shelves. photos/2026/ is part of the key. The console shows it as a folder, but there is no folder to create, rename or set permissions on.
• You cannot open a box and change one page. Updating an object means writing the whole object again.
• Losing the label is not losing the box, if versioning is on. A delete then adds a marker on top, and the previous version can still be restored.

Buckets, objects and keys

• An object’s key is up to 1,024 bytes of UTF-8 and is case-sensitive: Report.csv and report.csv are different objects.
• A bucket lives in one AWS Region, which you choose when you create it.
• An account can create up to 10,000 general purpose buckets by default; more is a quota request.
• Reads after a write are strongly consistent: once a PUT or DELETE succeeds, the next read sees it.

Bucket names

A general purpose bucket name is 3 to 63 characters of lowercase letters, numbers, periods and hyphens. It begins and ends with a letter or number, has no two adjacent periods, is not shaped like an IP address and does not start with xn--. Avoid periods unless the bucket hosts a website.

Names live in a namespace shared by every AWS account in the partition. Once you delete a bucket, anyone can create one with the same name, and anything still pointing at the old name would then reach theirs. AWS now offers an account regional namespace, which only your account can create names in. Such names end with your account ID, the Region and -an, for example amzn-s3-demo-bucket-111122223333-us-west-2-an. AWS recommends it for new buckets.

Storage classes, versions and encryption

Every object has a storage class, which trades price against how fast and how often you can read it:

• Versioning keeps every version of every object. A bucket starts unversioned; once you enable versioning it can be suspended but never returned to unversioned.
• Encryption is on for everything: since 5 January 2023 every new object is encrypted at rest with S3-managed keys (SSE-S3) at no extra charge. Choose SSE-KMS for keys you control and audit, and remember what it does to public access below.

Who can read what

Four controls decide access, and the defaults on a new bucket keep it private:

1. IAM policies on users and roles in your account. Within one account, an Allow here or in the bucket policy is enough, and an explicit Deny in either wins. Across accounts, both must allow.
2. The bucket policy, a resource-based policy with a Principal. This is the way to grant access to other accounts, AWS services or, deliberately, everyone.
3. ACLs, the old per-object grants. New buckets use Bucket owner enforced Object Ownership: ACLs are disabled, no longer affect access, and every object belongs to the bucket owner.
4. Block Public Access, four settings at the account (or organization) and bucket level. S3 applies the most restrictive combination. Two of them refuse new public grants; the other two stop existing ones working.

“Public” has a precise meaning. An ACL is public if it grants anything to the AllUsers or AuthenticatedUsers groups. A bucket policy is public unless every statement is limited to fixed values, such as specific principals, accounts or IP ranges, with no wildcard.

Is this object public?

Set up a bucket the way yours is, and the explorer says whether an anonymous request from the internet can read an object in it, and which setting decides. It runs in your browser.

Left out: access points, VPC endpoint policies, policies with conditions (one limited to fixed IP ranges is not public in S3’s sense), Requester Pays, and objects owned by another account in a bucket with ACLs enabled. IAM Access Analyzer for S3 reports which buckets are public in your account.

Sharing without going public

A presigned URL lets one person download (or upload) one object for a limited time, with no change to the bucket’s permissions. Anyone holding the URL can use it until it expires, so treat it like a password.

The URL carries the permissions of the credentials that signed it, and lasts no longer than they do. The CLI’s default is an hour and its maximum 604,800 seconds (7 days); a URL signed with a role’s temporary credentials expires when that session does.

Static websites

S3 can serve a bucket as a website, but the website endpoint speaks only HTTP, and serving it directly means making the bucket public. The better pattern is an Amazon CloudFront distribution in front of a private bucket, using origin access control: you get HTTPS and a custom domain, and Block Public Access stays on. It is also the only way to serve SSE-KMS-encrypted objects to the public, because SSE-KMS does not support anonymous users.

Try it with the AWS CLI

A private bucket, an upload, a presigned link and a check that nothing is public. These commands were checked against the AWS CLI’s own help (version 2.32.5); they were not run against a live account for this page. The bucket name includes your account ID and a random number, because names are shared across accounts.

Cleaning up deletes data. aws s3 rb --force deletes every object and then the bucket; it cannot delete old versions, so a versioned bucket must be emptied version by version first.

Troubleshooting 403 Access Denied

Read the whole message first: within one account or organization, S3 names the type of policy that refused the request. Then confirm who is asking, and only then change policies.

Production practice

• Leave Block Public Access on at the account level. AWS recommends all four settings for the account and every bucket; turn one off only for a bucket that must be public, and know why.
• Keep ACLs disabled. Grant access with IAM and bucket policies, which can be read and reviewed in one place.
• Serve public content through CloudFront with origin access control rather than a public bucket.
• Turn on versioning for data you cannot re-create, with lifecycle rules so old versions do not grow forever.
• Choose SSE-KMS where you need to control and audit key use, and give the readers kms:Decrypt.
• Prefer the account regional namespace for new buckets, so a deleted bucket’s name cannot be taken by someone else.

Common mistakes

• Turning off Block Public Access on the bucket and wondering why it is still private: the account-level settings still apply.
• Granting s3:GetObject on arn:aws:s3:::bucket instead of arn:aws:s3:::bucket/*.
• Setting a public ACL on a bucket whose ACLs are disabled, and expecting it to do something.
• Encrypting public website files with SSE-KMS.
• Treating photos/ as a folder with its own permissions.
• Deleting a bucket whose name other systems still reference.

Interview questions

How do you make an S3 bucket readable by the public, safely?

Usually you do not: you put CloudFront with origin access control in front of a private bucket. If it must be public, turn off the relevant Block Public Access settings for that bucket only and grant s3:GetObject on the objects in a bucket policy.

What is the difference between BlockPublicPolicy and RestrictPublicBuckets?

BlockPublicPolicy refuses saving a public bucket policy. RestrictPublicBuckets stops an existing public policy from working, limiting the bucket to AWS services and the owner’s account.

Why might an object be unreadable to anonymous users even with a public policy?

Block Public Access at the account level, or SSE-KMS encryption, which does not support anonymous users.

How do you give a customer a file without making anything public?

A presigned URL, with an expiry no longer than needed, signed by credentials that will last that long.

Questions people ask

Is S3 a file system?

No. It stores whole objects by key over HTTPS. Tools can make it look like a file system, but there is no partial write, rename or lock.

Can I rename a bucket?

No: a bucket’s name, owner and Region are fixed when it is created. Create a new bucket, copy the objects across, then delete the old one.

Why can I not create a bucket called “backup”?

Names are shared by every account in the partition, so short names are long gone. Use a name with your account ID in it, or the account regional namespace.

Where to go next

• Every AccessDenied phrase decoded, and an explorer for the IAM side: AWS IAM explained.
• The network S3 traffic leaves through, and gateway endpoints that keep it off the NAT bill: Amazon VPC explained.
• The instances that read and write it, with roles rather than keys: Amazon EC2 explained.
• Checking a policy document before you apply it: the YAML and JSON converter validates the JSON.