Published on

Amazon S3 Cheat Sheet

Authors

AWS S3 Ultimate Cheat Sheet: A Complete Guide for 2025

Introduction

Amazon Simple Storage Service (S3) is a scalable, high-speed, web-based cloud storage service designed for online backup and archiving of data and applications on AWS. This comprehensive cheat sheet covers everything you need to know about S3, from basic concepts to advanced features and best practices.

S3 Fundamentals

Core Concepts

  • Bucket: Container for objects stored in S3
  • Object: File and any metadata that describes the file
  • Key: Unique identifier for an object within a bucket
  • Storage Capacity: Virtually unlimited

Bucket Properties

  • Each bucket name must be globally unique across all AWS accounts
  • Names must be DNS-compliant (3-63 characters)
  • Bucket names cannot be changed after creation
  • Default limit of 100 buckets per AWS account
  • Region cannot be changed after bucket creation
  • Can host static websites

Data Consistency Model

  • Strong read-after-write consistency for all operations
  • Strong consistency for overwrite PUTS and DELETES
  • Eventual consistency for listing all buckets after deletion

S3 Storage Classes

Frequently Accessed Data

  • S3 Standard: General-purpose storage for frequently accessed data
  • S3 Express One Zone: High-performance, single-AZ storage for latency-sensitive applications (10x faster than S3 Standard)

Infrequently Accessed Data

  • S3 Standard-IA: Long-lived, less frequently accessed data, multiple AZs
  • S3 One Zone-IA: Less expensive option, data stored in single AZ
  • Both require objects larger than 128KB and minimum 30-day storage

S3 Intelligent-Tiering

  • Automatically moves data between access tiers based on usage patterns
  • Tiers: Frequent access, infrequent access, archive, deep archive
  • No retrieval fees
  • Objects move to infrequent tier after 30 days of no access

Archive Storage Options

  • S3 Glacier Instant Retrieval: Millisecond retrieval for rarely accessed data
  • S3 Glacier Flexible Retrieval: Low-cost archive storage with retrieval times from minutes to hours
  • S3 Glacier Deep Archive: Lowest-cost storage for long-term retention, 12+ hours retrieval time

S3 On Outposts

  • Object storage on AWS Outposts for on-premises environments
  • Encrypted with SSE-C and SSE-S3
  • Supports lifecycle rules and access points

S3 API and Access Methods

API Types

  • REST API: Standard HTTP requests for bucket/object operations
  • SOAP: Support deprecated over HTTP, still available over HTTPS

URL Access Patterns

  • Virtual Hosted-Style: http://bucket.s3.amazonaws.com
  • Path-Style: http://s3.amazonaws.com/bucket
  • Transfer Acceleration: bucket.s3-accelerate.amazonaws.com

Bucket Configurations

SubresourceDescription
locationSpecify the AWS Region for bucket creation
policy & ACLGrant and manage bucket-level permissions
corsConfigure cross-origin resource sharing
websiteConfigure bucket for static website hosting
loggingTrack requests for access to your bucket
event notificationConfigure bucket to send notifications
versioningPrevent accidental deletion or overwrite
lifecycleDefine rules for objects with defined lifecycle
cross-region replicationAutomatic copying across different regions
taggingStore and manage tags for cost allocation
requestPaymentSpecify who pays for downloads
transfer accelerationFast, secure transfers over long distances

S3 Objects

Object Properties

  • All objects are private by default
  • Each object has data, a key, and metadata
  • Object metadata cannot be modified after upload
  • Maximum single PUT operation: 5GB
  • For objects >5GB: use multipart upload (up to 5TB)
  • Up to 10 tags per object

Object Operations

  • Delete operations in versioned buckets add delete markers
  • S3 Select allows retrieving partial object data
  • S3 Object Lambda supports HeadObject, ListObjects, and ListObjectsV2 operations

S3 Security

Access Control Methods

  • Bucket Policies: JSON-based access policies (limited to 20KB)
  • IAM Policies: User-based permissions
  • Access Control Lists (ACLs): Legacy method, limited to 100 permissions
  • Access Points: Named network endpoints with dedicated access policies

Encryption Options

  • Server-Side Encryption:
    • SSE-S3: Amazon S3-managed keys
    • SSE-KMS: AWS KMS-managed keys
    • SSE-C: Customer-provided keys
  • Client-Side Encryption:
    • Using AWS KMS customer-managed key
    • Using client-side master key

Additional Security Features

  • MFA Delete: Requires multi-factor authentication for deletion
  • VPC Endpoints: Private connections from VPC
  • Object Lock: Prevents deletion for fixed time or indefinitely
  • Access Analyzer: Identifies unintended public access

Data Management Features

Versioning

  • Protects against unintended overwrites and deletions
  • Disabled by default, must be explicitly enabled
  • Once enabled, can only be suspended, not disabled
  • DELETE operations add a delete marker instead of removing objects

Lifecycle Management

  • Transition Actions: Move objects between storage classes
  • Expiration Actions: Delete objects after specified time
  • Automatic application to object groups based on rules

Cross-Region Replication (CRR)

  • Requirements:
    • Both buckets must have versioning enabled
    • Buckets must be in different regions
    • Proper IAM permissions
  • What's replicated:
    • New objects created after configuration
    • Object metadata and tags
    • Encrypted objects (with configuration)
  • What's not replicated:
    • Existing objects before configuration
    • SSE-C encrypted objects
    • DELETE operations with version ID

Monitoring and Management

Monitoring Tools

  • CloudWatch Metrics: Track storage, requests, and performance
  • CloudTrail: Log API activities
  • S3 Storage Lens: Analytics for storage optimization
  • Inventory Reports: Audit and report on objects and metadata

Batch Operations

  • Manage billions of objects with a single request
  • Copy objects between buckets
  • Set access controls or tags
  • Restore objects from Glacier

S3 Website Hosting

Configuration

  • Enable website hosting in bucket properties
  • Set index and error documents
  • Configure public read permissions via bucket policy

Limitations

  • Supports only publicly readable content
  • Only GET and HEAD requests
  • No SSL support for website endpoints

S3 Pricing Components

Charge TypeDescription
StorageBased on object size, duration, and storage class
RequestsVaries by request type (GET, PUT, etc.)
RetrievalsFor IA and Glacier storage classes
Early DeletesFor objects deleted before minimum duration
Data TransferInbound free, outbound charged (with exceptions)
ManagementFeatures like analytics, inventory, etc.

Best Practices

  1. Security:

    • Use bucket policies and IAM roles instead of ACLs when possible
    • Enable default encryption for all buckets
    • Regularly audit permissions with Access Analyzer
  2. Performance:

    • Use Transfer Acceleration for large files or distant clients
    • Implement prefixes for high-request-rate workloads
    • Consider S3 Express One Zone for latency-sensitive applications
  3. Cost Optimization:

    • Implement lifecycle policies to transition objects
    • Use S3 Intelligent-Tiering for unpredictable access patterns
    • Enable Requester Pays for shared data access
  4. Data Protection:

    • Enable versioning for critical data
    • Set up cross-region replication for disaster recovery
    • Configure S3 Object Lock for compliance requirements

New S3 Features (2025)

S3 Metadata (Preview)

  • Automated, queryable metadata updated in near real-time
  • Stored in Apache Iceberg tables
  • Query with Athena, EMR, Redshift, and more

S3 Tables

  • Purpose-built storage for tabular data
  • Higher transaction rates and query throughput
  • Native Apache Iceberg format support
  • Automated optimization and maintenance

Storage Browser for S3

  • Open-source component for web applications
  • High-throughput data transfer
  • Integrates with React-based applications

Conclusion

Amazon S3 continues to evolve as the foundation of AWS storage services. Understanding its features and capabilities allows you to build secure, performant, and cost-effective cloud storage solutions. Whether you're storing static website assets, building data lakes, or archiving long-term records, S3 provides the flexibility and scalability to meet your needs.

Additional Resources