Space-Based Architecture and the In-Memory Data Grid

Every style that keeps a relational database at the centre of its request path inherits that database’s concurrency limit. Connection pools are bounded, row locks serialize conflicting updates, and a hot row becomes a queue regardless of how many application servers sit in front of it. Space-based architecture is the response that removes the shared store from the path entirely: application state lives in a distributed in-memory grid, requests are routed to the node that holds the relevant data, and the database becomes a persistence destination rather than a participant in each request.

The style descends from tuple-space coordination, in which processes communicate by writing and reading entries in a shared associative memory rather than by addressing each other. The modern form keeps the shared-memory abstraction and adds partitioning, replication, and a deployment manager that keeps the in-memory tier sized and available. Its trade is blunt: it spends durability and a large amount of operational complexity to buy read and write latency that a single database node cannot deliver.

The processing unit and the middleware around it

The style’s building blocks are processing units — self-contained application instances that hold their own data in memory and handle requests for that data — and a set of virtualized middleware components that the units share.

A messaging grid routes requests and events between units without the sender knowing which unit holds the state. A data grid partitions the in-memory data by key and keeps replicas. A processing grid dispatches work to units, and a deployment manager starts, stops, and rebalances units as capacity changes. The important structural point is that none of these are single instances: each is itself partitioned and replicated, because a single routing node would reintroduce the bottleneck the style exists to remove.

A request arriving for a customer’s shopping cart is hashed to a partition, and the unit that owns that partition handles it. No lock is taken, because no other unit touches that cart. Concurrency within the partition is still possible — two requests for the same key can arrive at the owning unit at the same time — so the unit serializes per key, either by accepting one request at a time for that key or by using a per-key compare-and-set. That is a narrow version of the single-message-at-a-time rule that actor mailboxes apply to an entire stateful process. The grid has narrowed the contention from the whole table to a single key; it has not eliminated it.

// routing is a pure function of the key; the replica set is chosen separately
int partitionFor(String key, int partitions) {
    int h = key.hashCode();
    h ^= (h >>> 16);                 // spread the high bits into the low ones
    return Math.floorMod(h, partitions);
}

The routing function must be stable across the cluster. If two nodes compute different partitions for the same key, requests reach a unit that does not hold the data and every such request becomes a remote fetch. A hash that depends on the node count, such as a modulo applied after rebalancing, moves nearly every key when capacity changes; consistent hashing or an explicit partition table moves only the affected ranges.

Replication decides the loss window

Each partition exists on more than one node, and the replication mode decides what a write acknowledgement means.

Synchronous replication acknowledges only after the configured number of replicas has the value. The write costs a network round trip to the replica and the loss window is zero for the acknowledged copies, but a replica that is slow or unreachable stalls writes for that partition. With two replicas and a synchronous requirement of both, a single node failure makes the partition unavailable for writes until a new replica is promoted, which is a worse outcome than losing the last few writes for many applications.

Asynchronous replication acknowledges from the primary and forwards the value to replicas in the background. Writes stay local and fast, and a primary failure loses whatever had not propagated — a window set by the replication step, so one network round trip on a healthy link, and unbounded when the replica is behind.

The intermediate arrangement, acknowledging from a quorum of replicas, is what most grids expose as a consistency level. The arithmetic is the standard one: with three replicas, a write quorum of two and a read quorum of two intersect, so a read sees the latest acknowledged write; a read quorum of one does not. Choosing the level per operation rather than per cluster is what makes the trade usable, since a balance display can tolerate a one-replica read and a payment authorization cannot.

Under a network partition, a grid that keeps accepting writes on both sides of the split produces two histories for the same key. Merging them after the partition heals requires a rule — last write by a version vector or a timestamp — and any rule discards one of the writes. A grid configured to refuse writes on the minority side avoids the merge at the cost of availability during the split. There is no configuration in which both survive.

Memory is the capacity limit, so eviction is part of the design

The database had a disk; the grid has a heap. Capacity is bounded by the total memory of the cluster divided by the replication factor, so three replicas of a 100 GB data set need 300 GB of cluster memory before overhead. Once the working set exceeds that, the grid must either reject writes or evict.

Eviction policies are the usual cache policies — least recently used, time-to-live, or a size-based rule — and each eviction has a consequence the request path feels. Evicting a value means the next read misses and must fetch it from the system of record, so the latency distribution becomes bimodal: hits are served from memory, misses include a database round trip that is orders of magnitude slower. A grid whose eviction rate is high is a cache with a distributed coherence protocol, and its throughput is bounded by the database after all, only with added hops.

Heap size interacts with this. A larger heap holds more of the working set and lengthens garbage collection pauses; a smaller heap avoids long pauses and evicts more often. Splitting the data across more nodes with smaller heaps is the usual resolution, which is why the deployment manager’s ability to add capacity matters as much as the grid’s own data structures.

Eviction also creates a correctness question. A value evicted while a write-behind flush is pending must not be lost, so the eviction path has to hand the pending write to the persistence mechanism rather than drop it with the value. A grid that treats eviction as deletion has a data loss mode that appears only under memory pressure.

Write-behind and the durability the style gives up

Most space-based systems persist to a database asynchronously. The unit acknowledges the client after the in-memory write and queues the change for a writer that batches it to the store. Batching is what makes the database keep up: a hundred single-row updates become one batch, and the database sees a fraction of the operations the grid handles.

The ordering of that batch is the subtle part. Two updates to the same key queued in order can be written in order by a single writer, but two writers flushing different partitions can write the same row in either order, and the database’s last write wins — which may be the older value. Persisting the version alongside the value and rejecting a write with an older version is the guard, and it requires the database schema to carry that column. Without it, the store can end up behind the grid permanently, and a restart loads stale state that the grid then serves as current.

Acknowledging before the durable write sets the loss window to whatever is queued. For a batch interval in the low milliseconds, a node failure loses that interval’s writes for the partitions it owned; a full cluster failure loses everything since the last flush. This is acceptable for session state and for derived data that can be recomputed. It is not acceptable for a ledger, and the style is a poor fit for anything whose loss would require reconciliation with an external party.

Idempotent replay closes part of the gap. If the flush is redelivered after a failure, applying the same version twice must be harmless, which the version guard provides. Snapshots of the grid at a point in time give a faster restart than replaying the whole store, at the cost of the storage and the coordination needed to take one consistently across partitions.

Hot keys, rebalancing, and the cost of a node leaving

A single key that receives a disproportionate share of traffic defeats the partitioning. The unit owning that key serializes the requests and every other unit is idle for that workload. Replication does not help, because the replicas serve reads only if the consistency level permits stale reads, and writes still go to the primary. The remedies are to split the hot key into sub-keys that can be updated independently and merged on read, or to batch updates to it at the owner. Both change the data model, which is why a hot key discovered after launch is an expensive defect.

Losing a node triggers rebalancing: the partitions it owned are reassigned, replicas are promoted, and a new replica is built from the surviving copy. The process moves data proportional to the affected partitions, and during the move reads may be served from a replica that is behind. Recovery time is therefore a function of partition size and network bandwidth, not of the number of nodes, and a grid with very large partitions recovers slowly no matter how much hardware is available. Smaller partitions make rebalancing cheaper and increase the metadata the routing layer must maintain.

The style’s operational surface is the price: a cluster with its own membership protocol, its own consistency configuration per operation, its own persistence pipeline, and its own recovery behaviour. It is justified where the request rate genuinely exceeds what a shared database can serve and where the data fits in memory at a cost the business accepts. Where the working set is small or the load is moderate, the same layered application over a database delivers the behaviour with one system to operate instead of four.