Incremental Document Synchronization Between Client and Server

A language server is asked about a document it did not parse from disk and cannot open reliably: the editor’s buffer may contain unsaved changes, an overridden encoding, or a language inferred from the first line rather than from the extension. The protocol therefore makes the client responsible for the server’s copy of the text, and specifies how that copy is transferred. The transfer is also the protocol’s most fragile channel, because it is stateful in both directions and because a single misapplied edit produces a server model that is wrong forever, in ways that only show up as nonsense results much later.

What the client promises when a document opens

Synchronization concerns documents that are open in the editor, not files on disk. Opening a document triggers a notification carrying its URI, the language identifier the client assigned, a version number, and the full text. Closing triggers the corresponding notification, after which the server must forget the document: further feature requests for that URI are outside the contract, and any cached analysis must be dropped, because the next open may carry different content.

Saving is separate from changing, and is optional. Save notifications may be sent before the write, after it, or not at all, and a pre-save notification can carry a request for edits that should be applied before saving — the mechanism used for actions such as trimming whitespace or organizing imports. Since disabling that step is a client decision, a server cannot rely on its edits being applied.

Everything else — completion, hover, diagnostics, tokens — assumes the server holds an accurate copy. The synchronization mode is advertised once, as part of the server’s capabilities, and it is an enumerated choice: no synchronization at all, full, or incremental. Omitting the property yields no synchronization, which is the most common silent failure of a new server, since the process starts, answers nothing useful, and never reports an error.

Full versus incremental, and what each costs

Full synchronization sends the entire document text with every change. It is idempotent by construction: the server replaces its copy wholesale, so a lost, reordered, or duplicated notification leaves it wrong only until the next one arrives, and no range arithmetic is involved. The cost is bandwidth proportional to document size per keystroke. For a file of a few thousand characters on a local socket this is invisible; for a megabyte of generated source, each keystroke moves a megabyte, and the server re-hashes and re-parses it or maintains a content key to notice that the change was small.

Incremental synchronization sends a description of what changed: a range in the old text and the text that replaces it. A keystroke becomes a message of tens or hundreds of bytes regardless of document size, and the range is expressed in the position encoding negotiated during the handshake, which ties this channel to Byte Offsets, UTF-16 Code Units, and Editor Positions. The price is that the server must apply the change to an exact replica: any divergence in the replica, in the encoding of the range, or in the order of application corrupts the mirror, silently and persistently.

A middle path used by some servers is incremental synchronization with per-change validation: the server verifies that the text being replaced matches what it expects, and on mismatch requests a full resynchronisation. The protocol has no dedicated request for that, so the practical signals are a notification carrying the whole document as a single change with no range, or a close and re-open of the document by the client when the server reports that its state is unusable.

Change ranges are sequential, not simultaneous

A change notification carries a version and a list of changes, and the list is not a set of edits against one state. Each change in the list is computed against the document state produced by applying all preceding changes in the same list: the first applies to state S and yields S’, the second applies to S’ and yields S”. Applying them in any other order, or resolving every range against the original text, produces a different document whenever the ranges overlap or when shifts move text that a later range refers to.

A change is either a replacement with a range — start and end positions plus the new text — or, in full mode, the whole text with the range omitted. Multi-cursor editing and formatters typically produce several changes in one notification, which is why the sequential rule exists rather than requiring clients to coalesce edits. The protocol also defines a deprecated field giving the length of the replaced range, retained for compatibility; it is expressed in the same unit as the positions and is unsafe to rely on for text containing surrogate pairs, which is one reason the replacement text and positions are now the only inputs implementations should read.

def apply_changes(text: list[str], changes: list[dict], to_index) -> list[str]:
    """text is a list of lines; to_index maps (line, char) -> (row, col)."""
    for change in changes:
        if "range" not in change:                 # full-document replacement
            text = change["text"].splitlines(keepends=True)
            continue
        (sl, sc), (el, ec) = to_index(change["range"]["start"]), \
                             to_index(change["range"]["end"])
        before, after = text[:sl], text[el + 1:]
        head = text[sl][:sc]
        tail = text[el][ec:] if el < len(text) else ""
        inserted = change["text"].splitlines(keepends=True)
        if not inserted:                          # pure deletion
            joined = head + tail
            text = before + ([joined] if joined else []) + after
        else:
            inserted[0] = head + inserted[0]
            inserted[-1] = inserted[-1] + tail
            text = before + inserted + after
    return text

The version on the notification identifies the state after all listed changes, not before them, so a server comparing versions must compare against the version it will hold once the list is applied. Version numbers increase monotonically per document while it is open, and a server that receives a version lower than the one it holds has received an out-of-order or duplicate notification.

Versions, gaps, and the resync that does not exist

Version numbers are useful for one purpose: detecting that notifications were lost or reordered. They are not a mechanism for recovery, because nothing in the protocol lets a server ask for the text at a particular version or for the changes it missed. A gap in versions is an unrecoverable divergence in incremental mode, and the only honest responses are to mark the document’s analysis unusable until a full-content change arrives, or to treat the document as closed and wait for a fresh open.

This asymmetry explains a design that looks wasteful: clients that permit it send the whole document on open and thereafter send incremental changes, while clients with cheap bandwidth and simple servers may configure full synchronization for every change. The bytes saved by incremental mode are real — a single keystroke in a large generated file becomes a few hundred bytes instead of the entire file — but they are worth nothing if the applying code has a defect, and the defect will not announce itself.

Notifications, ordering, and threading

Two ordering guarantees make mirroring possible, and both must be respected by the receiving server. Notifications arrive in the order the client sent them, because they travel on one ordered stream, so applying them in receive order reproduces the client’s history exactly. Within a notification, the changes apply in list order. Nothing else is guaranteed: the protocol deliberately allows a server to answer requests out of order, and a server is free to process requests on several threads.

The interaction is the trap. A server that dispatches every incoming message to a thread pool can apply change notifications concurrently, and then the mirror depends on scheduling. The fix is per-document serialisation: requests may run concurrently, but changes for a given document must be applied in receive order, and a request that reads the document must either take a snapshot or declare the version it read. Servers that omit the snapshot produce results against a document state that no longer exists, which is the situation the content-modified error exists to report rather than to hide.

Where mirrors diverge

Divergence has four common causes. The first is a range computed in the wrong unit, which shifts every subsequent position by the difference in width between the encodings. The second is line-terminator disagreement: a client that reports positions computed over CRLF text while the server splits lines on LF alone sees half as many lines and misplaces every range below the first few. The third is applying a batch of changes against one state instead of sequentially, which shows up only when a batch contains overlapping edits. The fourth is a text edit from the server applied by the client to a different version of the document, which corrupts the client — the same problem in the opposite direction, and the reason server replies carry a document version that the client can compare before applying anything.

Detecting divergence early is cheaper than recovering from it. A server that keeps a hash of its mirror per document and a client that logs version gaps will locate the first bad edit rather than the thousandth symptom, and where the mirror cannot be trusted, the correct recovery remains the crude one: discard the state and take the full text again. The features that sit on top of the mirror, beginning with the diagnostic channel described in Diagnostics Delivery: Push, Pull, and Stale Results, inherit whatever accuracy it has.