What is Amazon S3 Batch Operations?

• It allows you to perform batch operations on large sets of objects in Amazon S3.
• You can select objects based on prefixes, tags, and metadata.
• Actions that can be performed include copying, archiving, applying access controls, and adding object tags.
• The goal is to simplify and automate repetitive tasks.

Any news on Batch Operations?

• You can now manage objects within an entire bucket, based on prefix, suffix, creation date, or storage class.
• You can quickly apply the operation to all the matching objects.

Benefit of Batch Operations

• You can easily perform one-time or batch workloads:
• Copying objects between staging and production buckets
• Invoking an AWS Lambda function to convert file types
• Restoring archived backups from S3 Glacier storage

Create a replication rule

Set the filtering: file name prefix

Create a CSV inventory configuration for batch operation

Manifest.csv contains all objects' names for the batch operation

Create an S3 batch operation using a CSV inventory configuration

Select "copy" to replicate objects to the destination bucket.

Create an S3 batch operation using an S3 replication configuration

Filter objects by creation date
You can specify filters to reduce the scope of replicated objects.
These filters work in conjunction with existing filters in your replication configuration.
If no filters are specified, all objects defined by the replication configuration will be replicated.

Full visibility

Monitor the running time and percentage of objects completed.
Receive a detailed completion report with the status of each object.

Reference

Amazon S3 Batch Operations now manages buckets or prefixes in a single step

Replication configuration

Replication configuration hand-on