Storage Tiers Are a Bet on Access Frequency: Minimum Durations and Retrieval Cost

The tier is a pricing schedule, not a device

A storage class does not name a different kind of hardware. Every class stores bytes on the same durable substrate, with the same replication target and the same durability claim. What changes is the schedule of charges attached to the object: a per-byte-month rate for storage, a per-request rate for operations, a per-byte rate for retrieval, and a minimum period for which the storage charge applies regardless of how long the object actually lived.

The tiers on the major services sort along one axis. At the top, a general-purpose class bills storage at the highest per-byte rate, charges nothing for retrieval, and answers reads in milliseconds. Below it, infrequent-access classes cut the storage rate substantially, add a per-byte retrieval charge, and bill a minimum object size, 128 KB on the S3 classes, so that a 4 KB object is charged as 128 KB. Below those, archival classes drop the storage rate by a large factor at each step, add a minimum storage duration, and add either a retrieval fee with restored-object latency or a rehydration delay measured in minutes to hours.

The tier is therefore a wager: that the object will be read rarely enough that the retrieval charges plus the minimum-duration exposure are smaller than the storage savings. The wager is computable, and it is worth computing before a lifecycle rule automates it across a bucket.

Minimum duration converts a cheap class into an expensive one

Minimum storage durations are the mechanism that makes the wager risky. The S3 infrequent-access classes carry a 30-day minimum; the archival classes carry 90 days for the instant-retrieval and flexible-retrieval tiers, and 180 days for the deepest archive tier. The charge is not a penalty line item: deleting or transitioning an object before the minimum elapses bills the remaining days at the archival rate.

The effect on short-lived data is severe. An object written into a 180-day tier and deleted after one day is billed for 180 days of archival storage, which at the archival rate can exceed the cost of keeping it in the general class for its actual lifetime. A pipeline that writes transient intermediate results into an archival tier and cleans up nightly pays the full minimum on every one of them, and the saving that motivated the transition inverts.

The same arithmetic applies to transitions rather than deletions. Moving an object from a 180-day class to a colder one at day 60 bills the remaining 120 days at the old class’s rate while the object also begins accruing charges in the new one. A lifecycle policy that cascades objects through several tiers on fixed day counts can therefore bill more than leaving them in the middle tier, and the cascade must be designed so that each hop occurs after the previous class’s minimum has elapsed.

Retrieval charges invert the comparison for read-heavy data

Storage price is the visible number; retrieval price is the one that scales with use. In the infrequent-access classes the retrieval charge is per byte read, and in the archival classes it is both per byte and, for the flexible tier, per request, with an additional charge for the restore operation that copies an archived object into a readable form before it can be fetched.

Consider a dataset of S bytes read n times per month in full. The monthly cost in a general class is the storage rate times S. In an infrequent class it is a lower storage rate times S plus a retrieval rate times S times n. The crossover is at the point where the storage saving per byte equals the retrieval charge per byte read, and it depends only on n, not on S. A dataset read once a quarter stays cheap in the colder tier; a dataset read weekly often does not, and a dataset read daily almost never does.

The archival tiers add a constraint that no amount of arithmetic fixes: an archived object cannot be read at all until it is restored, and restoration for the flexible tier takes minutes to hours depending on the retrieval option chosen. Workloads with an interactive read path cannot use those tiers regardless of price, and workloads that need a guaranteed restore time must use the faster retrieval option, which is priced higher per byte.

A short script makes the crossover explicit for the simple two-term case:

def reads_to_break_even(storage_saving_per_gb_month, retrieval_per_gb):
    """Full reads per month at which retrieval cost cancels the storage saving."""
    return storage_saving_per_gb_month / retrieval_per_gb

# Supply a provider's published per-GB rates; the shape of the result is the point.
# e.g. a 0.01 saving against a 0.004 retrieval charge breaks even at 2.5 reads/month.
print(reads_to_break_even(0.01, 0.004))

The function takes the two rates a price list supplies and returns the number of full reads per month at which the two charges are equal. Below that number the colder class is cheaper, above it the general class is. Because the retrieval charge is per byte, a dataset read in part moves the threshold in proportion to the fraction read.

What the transition itself costs

Transitioning an object is an operation, and operations are billed. A lifecycle rule that moves a million small objects to a colder class issues a million transition requests, each charged at a rate that is well above the rate for a plain read. For small objects the transition charge can exceed the storage saving for the whole first month, and for objects below the minimum billable size it can exceed the saving for the object’s entire lifetime.

This is the interaction that makes object size, not just access frequency, part of tier selection. An archival tier is economical for objects in the hundreds of megabytes and above, where one transition request is amortised over a large saving. It is uneconomical for a bucket of 20 KB log fragments, where the per-object minimum billable size of the infrequent class already inflates the storage charge and the transition count multiplies the request charge.

Compaction is the usual answer: aggregate small objects into larger ones before transitioning, so that the tier change applies to a few large objects rather than many small ones. That restructuring is itself a copy, since objects are immutable and cannot be concatenated in place; the mechanics of assembling a larger object from existing bytes belong to multipart upload part sizing on the write side and to byte-range reads and parallel fetch on the read side.

Lifecycle rules are evaluated as a set, in a fixed order

A lifecycle configuration is a list of rules, each with a filter selecting objects by prefix, tag, or size, and each with a set of actions. The actions include transitioning the current version to a class at a given age, transitioning noncurrent versions at a given age after they stopped being current, expiring the current version, expiring noncurrent versions, and aborting incomplete multipart uploads.

Rules do not execute as a script; the service evaluates the applicable actions for each object against its age and state, and the documented ordering matters. Transition and expiration are separate actions, and an object that becomes eligible for both on the same day is subject to both, with expiration winning in effect, so the object is deleted rather than transitioned and then deleted. A rule that transitions to an archival class at day 30 and expires at day 31 therefore produces one day of archival billing plus, if the archival class has a minimum duration longer than a day, the remainder of the minimum as a charge on a deleted object. The minimum-duration mechanics above are what make that pattern expensive rather than merely pointless.

Noncurrent versions need their own rules. Enabling versioning, as described in versioning and delete markers, means an overwritten object leaves its previous bytes in place indefinitely; a lifecycle rule that transitions or expires only current versions will never touch them. The days count for a noncurrent version starts when it stops being current, not when it was written, which is the detail that makes a “expire after 30 days” rule safe for a long-lived key that is rewritten frequently.

Incomplete multipart uploads are the third category, and they are invisible to object listings. An abort rule keyed on the number of days since initiation is the only automated way to reclaim those parts, and its absence is a common source of storage that grows without any corresponding object growth.

Choosing a class from the access pattern, not the name

The defensible procedure is to measure three quantities per dataset: the fraction of bytes read per month, the acceptable latency for a read, and the expected lifetime of each object. A dataset read at less than roughly one full pass per month, tolerant of millisecond latency, and living longer than a month fits an infrequent-access class. A dataset read a few times a year, tolerant of hours of restore latency, and living longer than half a year fits an archival class. A dataset read continuously, or one whose objects are small and short-lived, belongs in the general class no matter how cold it looks by name.

Two things break the procedure. The first is a dataset whose access pattern changes without warning, which is exactly the case a tier bet is worst at handling; the retrieval charges arrive before the lifecycle rule can be revised, and revising a rule does not retroactively move objects back, it issues another set of transition requests. The second is a dataset whose reads are large but rare, such as a full-table scan once a quarter, where the per-byte retrieval charge is proportional to the whole dataset and the storage saving is proportional to it as well, so the comparison is decided entirely by the ratio of the two rates rather than by the frequency. Both cases argue for keeping the transition decision reversible and for treating a lifecycle policy as a priced decision rather than a default.