String Encoding and Buffer Ownership at the FFI Boundary

A string is the most common thing two languages exchange and the least well specified. It has at least three independent properties that must match before a single character is interpreted correctly: the encoding of the code units, the way the end of the string is marked, and the allocator that produced the memory. A binding can get all three right and still leak, because a fourth property, who is responsible for freeing the buffer, is not part of the value at all.

C makes the first two properties implicit and the third explicit. A char * is a pointer to bytes, terminated by a zero byte, with no associated length and no declared encoding. The compiler does not check that the buffer is large enough, that the terminator exists, or that the bytes form valid text. Every C function that accepts a string accepts an unbounded promise from its caller.

Encoding is a separate agreement from termination

Most modern language runtimes store text as UTF-8 in contiguous memory, which makes the encoding half of the problem smaller than it once was. A runtime that stores UTF-16, such as one following the Windows API convention, must transcode on every call, and the transcode allocates. A runtime that stores an abstract sequence of code points must decide what to do with ill-formed input, because UTF-8 permits byte sequences that no code point maps to.

RFC 3629 constrains the encoding usefully: a UTF-8 sequence is at most four bytes, the shortest form for a code point is required (overlong encodings are ill-formed), surrogates U+D800 through U+DFFF must not appear, and the maximum code point is U+10FFFF. A decoder that enforces those four rules rejects every ill-formed sequence; a decoder that only checks continuation-byte patterns accepts overlong forms and surrogate halves and produces a value no Unicode consumer expects. Passing such bytes across a boundary into a runtime with a stricter decoder produces a failure on the far side, attributed to the wrong layer.

The termination convention interacts with the encoding. A zero byte is a valid code unit only for the character U+0000, which cannot appear in well-formed text, so NUL termination and UTF-8 coexist. But it means a string cannot contain an embedded NUL, and a runtime whose strings are length-counted can hold one. A binding that converts a length-counted string containing an embedded zero into a C string will truncate it silently at the boundary, and the truncation is not detectable by the C side because the C side has no length to compare.

Length counting itself has a unit ambiguity. A C caller passing (char *buf, size_t len) means bytes. A Windows API taking a character count means UTF-16 code units. A runtime’s string length in its own representation may mean bytes, code units, or code points. Mixing them produces an off-by-a-factor error that is not always caught by a bounds check: passing a UTF-16 unit count where a byte count is expected under-allocates by half, and passing a code-point count where a byte count is expected under-allocates by up to four.

Three ownership models and what each requires

The first model is caller-owned buffers. The callee writes into memory the caller allocated and never frees it. This is the safest model because the allocator identity never crosses the boundary: the caller allocates and frees with its own allocator, and the callee only writes. The cost is that the caller must know the required size in advance.

The second is callee-allocated, caller-freed. The callee allocates with its own allocator and returns a pointer; the caller must free it with the matching deallocator. This works only when the two sides share an allocator, which is not guaranteed. A static C runtime linked into a shared library may have its own heap on some platforms, and a garbage-collected runtime’s allocator cannot accept a foreign free. The failure is a heap corruption that appears at an unrelated allocation, described in allocator mismatch.

The third is callee-allocated, callee-freed, with an explicit release call. The callee returns a pointer plus an obligation, and the caller must invoke a specific function to discharge it. This is the only model that is correct across allocators that do not share a heap, and it requires the API to name the release function. A caller that uses the platform free instead will corrupt the callee’s heap, and nothing in the type of the returned pointer prevents it.

The distinction matters because the pointer type is identical in all three cases. char * does not say whether it points into a caller buffer, into callee-allocated memory, or into static storage that must not be freed at all. Static storage is the worst variant: a function that returns a pointer to a string literal or a thread-local buffer looks exactly like a function that returns fresh memory, and calling free on it aborts or corrupts the heap.

The two-call pattern and its race

When the required size is not known in advance, the conventional protocol is to call the function twice. The first call passes a null pointer and a zero length; the function returns the size it would need, including the terminator. The caller allocates that many bytes, calls again, and the second call fills the buffer.

#include <stdlib.h>

/* Returns the required size when buf is NULL. */
size_t format_path(char *buf, size_t len, const char *name);

char *make_path(const char *name) {
    size_t need = format_path(NULL, 0, name);
    if (need == 0) return NULL;

    char *p = malloc(need);            /* caller's allocator, caller's heap */
    if (!p) return NULL;

    size_t wrote = format_path(p, need, name);
    if (wrote == 0 || wrote > need) {  /* buffer was too small, or contract broken */
        free(p);
        return NULL;
    }
    return p;
}

The pattern has a well-known race when the size depends on mutable state. If the value being formatted can change between the two calls, the second call may need more space than the first reported, and a correct implementation must detect that and report failure rather than truncate. The return value of the second call is therefore not decorative: it is the only signal that the buffer was adequate. An implementation that returns the number of bytes written without indicating truncation converts a detectable error into silent data loss.

There is also an off-by-one class of bug specific to NUL-terminated output. If the first call reports the number of characters excluding the terminator, a caller that allocates exactly that many bytes has no room for the zero, and the callee must either truncate or write past the end. The convention must state whether the reported size includes the terminator, and both sides must agree. APIs that return the size including the terminator are safer for this reason; APIs that return a character count and require the caller to add one put the requirement in documentation that is routinely skipped.

Converting without copying, and when it is impossible

A zero-copy path exists when both sides agree on representation. If a runtime stores UTF-8 in a contiguous buffer with a known length, and the C function accepts a pointer and a byte length, the runtime can hand over a pointer into its own buffer without allocating, provided the buffer is not moved during the call. Runtimes that relocate objects during collection cannot make that promise without pinning, and pinning is itself a cost: it fragments the heap and prevents compaction for the duration.

The other obstacle is the terminator. A runtime that guarantees a trailing zero byte after its string data can pass a pointer directly to a function expecting a C string; one that does not must either copy or accept that the function will read past the end of the buffer looking for a zero that is not there. Adding the zero requires either spare capacity in the buffer or an allocation, so the “free” conversion is only free when the runtime’s representation was designed with the C convention in mind.

Transcoding is the case where copying is unavoidable. UTF-16 to UTF-8 can expand a string by up to fifty percent, so a buffer sized for the source is not necessarily large enough for the destination, and the two-call pattern must be applied in the destination’s units. Converting in the other direction can shrink, but a naive converter that allocates the source byte count as the destination size wastes memory and still needs the terminator accounted for.

Paths, locales, and the encoding that is not the encoding

File paths deserve separate treatment because the operating system’s encoding is not necessarily UTF-8. POSIX treats path names as byte strings with only two reserved values, the zero byte as terminator and the slash as separator; the bytes are interpreted according to the locale, and a name that is not valid in the current locale is still a valid name. A runtime that converts a path to UTF-8 for display and back to bytes for a system call can lose the original bytes if the conversion is lossy, and a file that cannot be reopened is the result.

The practical rule is that paths should cross the boundary as byte strings, not as text, and that any conversion to a display encoding should happen only where display is the purpose. On Windows the convention is UTF-16 and the API has separate narrow and wide entry points; the narrow ones interpret bytes in the process’s ANSI code page, which is a different mapping from UTF-8 on most systems and cannot represent every name. Calling the narrow entry point with UTF-8 bytes produces a path to a different file or a failure, and the failure is reported as “file not found” rather than “encoding mismatch”.

The general lesson is that “string” is not one type. Encoding, termination, length units, and ownership are four independent decisions, and an interface that specifies only the pointer has specified none of them. The stable C ABI approach is to make each decision explicit in the signature (an explicit length, a documented encoding, a named release function) so that a caller who reads the declaration has enough information to be correct. Strings are where that discipline pays most, because they are the argument that crosses the boundary most often and the one whose implicit assumptions are least visible.