Bitmap Memory and the Cost of Decoding at Full Resolution

A photograph stored as a 400 KB JPEG becomes, once decoded, a block of memory whose size is fixed by its pixel dimensions and its per-pixel format. The ratio between the two is often two orders of magnitude. An app that decodes images at their stored resolution and then displays them in a thumbnail has chosen to pay the larger number for no visible benefit, and the bill arrives as memory pressure and as decode time on the main thread.

The arithmetic before any code

Decoded size is width times height times bytes per pixel. A 12-megapixel image, roughly 4000 by 3000 pixels, at four bytes per pixel is 48 MB. At two bytes per pixel, 24 MB. The file it came from might be 3 MB. The compression ratio of the codec has nothing to do with the memory the decoded image occupies; the codec’s job ends when the pixels exist.

Four bytes per pixel is the common default because each pixel carries red, green, blue, and alpha in one 32-bit word, and the graphics pipeline wants that layout for blending. Two bytes per pixel is possible with a format that packs each channel into fewer bits, and the visual difference is often imperceptible for photographic content, but the format must be requested at decode time and the pipeline may need a conversion. Since the decoder runs on whichever thread asked for it, a full-resolution decode also spends part of the frame’s budget described in Frame Budget Arithmetic and the Real Deadline for Main Thread Work.

Alpha is worth a separate decision. If the image is fully opaque, storing an alpha channel costs a quarter of the memory for a constant value. The decoder will produce the alpha channel if the requested format includes it, and it will not if the format omits it. Choosing the format per image, rather than globally, is the single largest easy win in an image-heavy screen.

Why sampling is not the same as scaling

Decoding at reduced resolution requires the decoder to be told before it allocates. The JPEG codec supports this natively: a DCT-based image is stored as blocks of frequency coefficients, and taking only the low-frequency coefficients of each block produces a smaller image with almost no extra work. Decoding at one quarter linear scale costs roughly a quarter of the memory and less than a quarter of the time, because the inverse transform operates on fewer blocks.

The catch is that the scale factors a codec supports are not arbitrary. JPEG’s efficient subsampling is by powers of two per dimension, because that is what the block structure allows. Asking for a factor of three forces either a full decode followed by a resample, which saves memory but not time, or a decoder that does the resample in a second pass. Implementations therefore expose power-of-two sampling and expect the caller to pick the smallest power of two that still exceeds the target size.

Choosing the factor requires knowing the target size before decoding, which requires reading the image header without decoding the pixels. Every common format puts the dimensions in the header, and decoders expose a bounds-only pass that parses the header and returns width and height without allocating pixel storage. The correct sequence is: read bounds, compute the sample factor from the target and the bounds, decode once with that factor. Decoding twice, once to measure and once at scale, is the common mistake and doubles the work.

fun sampleFactor(srcWidth: Int, srcHeight: Int, reqW: Int, reqH: Int): Int {
    var factor = 1
    // Halve until the sampled image would still cover the requested size.
    while (srcWidth / (factor * 2) >= reqW && srcHeight / (factor * 2) >= reqH) {
        factor *= 2
    }
    return factor
}

The loop halves both dimensions together, which preserves aspect ratio and matches the power-of-two constraint. An image with an extreme aspect ratio can end up under-covered in one dimension; the condition requires both to still fit, so the result is a sample that is at least as large as requested in both axes.

Format conversion as a hidden cost

A decoder that produces a format the graphics pipeline cannot sample directly forces a conversion, and the conversion allocates a second buffer. A three-byte-per-pixel packed format converted to a four-byte format allocates 4/3 of the original plus the original, so the peak is more than twice the final size. Peak memory, not steady-state memory, is what triggers a kill.

The pipeline’s preferred format is usually a 32-bit word per pixel, and hardware-accelerated decoding often writes directly into a buffer the GPU can sample, avoiding both the conversion and the copy. Hardware decode is not universally available for every format and every size, and it has its own constraints: some decoders require dimensions to be multiples of a small block size, and some refuse images above a maximum dimension. When hardware decode is unavailable the software path runs, which is slower by a large factor and allocates its own working buffers.

The consequence for budgeting is that decode time is bimodal. The same image on two devices can take an order of magnitude apart, not because of CPU speed but because one has a hardware decoder for that format and the other does not. Any figure quoted for decode cost needs the path attached to it.

Caching decoded images

A decoded bitmap is expensive to produce and cheap to reuse, which makes it the natural unit of caching. The cache has to be bounded in bytes rather than in entries, because entries vary in size by three orders of magnitude. A cache limited to a hundred entries could hold 100 thumbnails or 100 full-resolution photographs, and the difference is the difference between a few megabytes and several gigabytes.

The bound should be a fraction of the memory the platform allows the process, not a fixed number. A device with a small memory class gives the app a small heap, and a cache sized for a large device will push a small device into the killer’s path. The platform exposes the memory class for exactly this reason, and the cache size should be derived from it rather than hardcoded.

Eviction needs to account for what is on screen. A least-recently-used cache will evict an image that is currently displayed if enough other images were touched after it, causing a re-decode and a visible flash. Pinning the images belonging to the visible viewport, and evicting only from the rest, avoids that. The same reasoning applies to the list-scrolling case: the working set is the visible rows plus a small margin, and anything beyond it is speculative.

Threading and the main thread

Decoding is CPU-bound and can take tens of milliseconds for a large image even at reduced scale. On the main thread that is a dropped frame or several. The decode therefore belongs on a background thread, with the result handed back to the main thread for display.

The handoff is where correctness problems appear. A decode started for a view that has since been recycled must be discarded; otherwise an image appears in the wrong row of a list. The standard solution is to tag each request with the identity of the view or the position it is for, and to check that tag when the result arrives. Cancellation is an optimization on top: a cancelled decode still consumes CPU, so a fast scroll that starts a decode per row will saturate the background pool with work whose results are all discarded. Bounding the pool and dropping requests that are no longer relevant is what keeps scrolling smooth.

There is a memory consequence too. Every in-flight decode holds a buffer, so a pool of eight concurrent decodes of full-resolution images holds eight full-resolution buffers at peak. Reducing concurrency reduces peak memory at the cost of throughput, and for images that are displayed one at a time the trade is usually worth taking.

Formats and sizes that break the simple model

Animated formats. A GIF or an animated WebP is a sequence of frames, and a decoder may hold several frames to composite the next one. The memory for an animation is not the memory of one frame but of the frames the decoder needs, plus any disposal buffers. Playing an animation therefore has a footprint that scales with the frame count, which is why large animated images are a common source of pressure.

Very large images. An image whose full-resolution decode exceeds the per-process heap limit cannot be decoded as a single bitmap at all, regardless of sampling, because the sampling happens inside the decoder and the decoder needs the output buffer. The remedy is tiling: decode a region at a time into a buffer small enough to fit. This is the mechanism behind deep-zoom viewers and it is not something a general-purpose image loader does automatically.

Colour profiles. An image with an embedded colour profile may be converted at decode time, and the conversion can allocate an intermediate buffer and take measurable time. Ignoring the profile produces wrong colours; honouring it costs memory and time. The decision is per-image and usually belongs to the loader rather than to the view.

Rotation metadata. A photograph may carry an orientation flag rather than rotated pixels. Honouring it requires either a rotated copy, which allocates a second full-size buffer, or a transform applied at draw time, which costs nothing in memory. The second is preferable and is what hardware-accelerated drawing paths support.

The memory arithmetic here feeds directly into the process ranking described in Memory Pressure, Kill Decisions, and What an App Can Observe Before It Dies: a screen that holds several full-resolution decodes is a screen that will be selected for termination first, and the kill will arrive without a stack trace to explain it.