GitHub - DotFox/transit.c: A data interchange format and set of libraries for conveying values between applications written in different programming languages.

A fast, zero-copy Transit reader and writer written in C11 with SIMD acceleration. One codec-agnostic engine speaks all three Transit wire formats — JSON, JSON-Verbose, and MessagePack — and decodes straight into a single arena with borrowed (never copied) string payloads.

TL;DR - What is Transit? Transit is a format and a set of libraries for conveying values between applications written in different languages. It is layered on top of JSON and MessagePack, so you get their tooling and speed, but with a much richer type system and built-in payload compression. Think of it as "JSON that round-trips real types":

Ground types from the host format: maps, arrays, strings, numbers, booleans, null Extension types JSON lacks: keywords :foo, symbols, instants (timestamps), UUIDs, URIs, big integers/decimals, characters, byte arrays, sets, and lists Built-in compression (caching): repeated map keys, keywords, symbols, and tags are written once and then referenced by a short ^N code, so verbose, key-heavy payloads shrink dramatically Extensible via tagged values: ship your own types over the wire with a tag plus a representation, and decode them with a custom handler — no more {"__type": "Date", "value": "..."} hacks Language-agnostic & self-describing: originally from Cognitect/Clojure, with implementations across many languages

Why Transit over plain JSON? A real type system (a keyword stays a keyword, an instant stays an instant), smaller payloads thanks to key caching, and first-class extensibility through tags — all while staying on top of the JSON/MessagePack infrastructure you already have. Learn more: Official Transit specification Features

🚀 Fast: SIMD-accelerated string scanning (SSE2 on x86-64, NEON on arm64), Grisu2 shortest-double formatting, and a memset-free tokenizer with inline integer parsing 💾 Zero-copy: transform-free strings, byte arrays, and keys are borrowed directly from the input buffer and threaded into the result tree — never copied 🧩 One engine, three formats:

a single codec-agnostic reader state machine and a single writer walk serve JSON, JSON-Verbose, and MessagePack; a format is just a token reader/writer behind one transit_codec_t 📡 Streaming emitter: a push API (transit_emit_*) that produces Transit incrementally without building a value tree, byte-identical to transit_write 🗜 Transit caching: ^N cache codes for repeated keys/keywords/symbols/tags, with reader and writer kept in lock-step 🏷 Rich type system: keywords, symbols, instants, UUIDs, URIs, bigint/bigdec, characters, byte arrays, sets, lists, and arbitrary tagged values 🔌 Custom handlers: decode your own composite tags into rich values at read time 🧹 Memory-safe: a single bulk arena backs each result tree — one transit_result_free() frees everything 📏 No recursion: the reader drives an explicit container stack, so there is no C-stack depth limit (tested to 60,000 levels of nesting) 🔧 Zero dependencies: pure C11 and the standard library only ✅ Conformance-tested: runs against the official cognitect/transit-format cross-implementation exemplar corpus 📦 Portable: SSE2/NEON SIMD with a strict-portability scalar fallback (NO_SIMD=1); builds as a static or shared library on Linux, macOS, and Windows

Table of Contents

Installation Quick Start Wire Formats API Reference

Core Functions Codecs Type System Scalar Accessors Collections Constructors Arena & Output Buffer Custom Read Handlers Options Errors Streaming Emitter

Examples Building Performance Project Status Notes Contributing Documentation License

Installation Requirements

C11-compatible compiler

(GCC 4.9+, Clang 3.1+, MSVC 2015+) Make (Unix/macOS) or CMake (Windows/cross-platform) Supported platforms:

macOS (Apple Silicon, Intel) — NEON/SSE2 SIMD Linux (arm64, x86-64) — NEON/SSE2 SIMD Windows (x86-64, arm64) — via MSVC/MinGW/Clang

Build Library Unix/macOS/Linux: # Clone the repository (with the exemplar corpus submodule) git clone --recurse-submodules https://github.com/DotFox/transit.c.git cd transit.c

# Build the static library (build/libtransit.a) make

# Run tests to verify the build make test Windows: # Clone the repository git clone https://github.com/DotFox/transit.c.git cd transit.c

# Build with CMake (works with MSVC, MinGW, Clang) .\build.bat

# Or use the PowerShell script .\build.ps1 -Test Integrate Into Your Project Option 1: Link the static library # Compile your code against the public header and archive cc -o myapp myapp.c -I/path/to/transit.c/include -L/path/to/transit.c/build -ltransit -lm

# Or add to your Makefile CFLAGS += -I/path/to/transit.c/include LDFLAGS += -L/path/to/transit.c/build -ltransit -lm Option 2: Include the source directly Copy include/transit.h and every file under src/ into your project and compile them together. Only include/transit.h is public; the src/ headers are internal. Quick Start #include "transit.h" #include <stdio.h> #include <string.h>

int main(void) { /* Transit-JSON for the map {:name "Alice" :age 30 :langs [:clojure :rust]}. In compact Transit-JSON a map is an array led by the "^ " marker and keywords are written as "~:name". */

const char *input = "[\"^ \",\"~:name\",\"Alice\",\"~:age\",30,\"~:langs\",[\"~:clojure\",\"~:rust\"]]";

/* Read Transit (zero-copy: payloads borrow from `input`). */ transit_result_t r = transit_read(transit_codec_json(), (const uint8_t *)input, strlen(input));

if (r.error != TRANSIT_OK) { fprintf(stderr, "read error at byte %zu: %s\n", r.position, r.message); return 1; }

printf("decoded a map with %zu entries\n", transit_count(r.value));

/* Look up the first value; the span borrows from `input`. */ transit_span_t name = transit_as_span(transit_map_val(r.value, 0)); printf("name: %.*s\n", (int)name.len, name.ptr);

/* One call frees the whole tree (a single arena). */ transit_result_free(&r); return 0; } Output: decoded a map with 3 entries name: Alice

Wire Formats The same value model and the same read/write algorithms drive all three formats; you pick one by passing the matching codec.

Codec Selector Encoding Caching Notes

JSON transit_codec_json() text yes Compact Transit-over-JSON; the default for interchange

JSON-Verbose transit_codec_json_verbose() text no Human-readable: native JSON objects for maps and RFC3339 instants

MessagePack transit_codec_msgpack() binary yes Compact binary Transit-over-MessagePack

Adding a new wire format means implementing a token reader + writer behind a transit_codec_t descriptor; the semantic layer never branches on format. API Reference The entire public surface lives in a single header: #include "transit.h" Core Functions transit_read() Read a value from a buffer using the given codec.

transit_result_t transit_read(const transit_codec_t *codec, const uint8_t *input, size_t len); Parameters:

codec: one of transit_codec_json(), transit_codec_json_verbose(), transit_codec_msgpack() input: the encoded bytes (must remain valid and unmodified for zero-copy reads) len: length of input in bytes

Returns: a transit_result_t (see Errors). On success, .error == TRANSIT_OK and .value holds the decoded tree. Important: free the result with transit_result_free(). String/bytes payloads in the tree may point directly into input, so the buffer must outlive the result. transit_read_opts() Like transit_read(), but with explicit options (verbose semantics, cache toggle, custom handlers). transit_result_t transit_read_opts(const transit_codec_t *codec, const uint8_t *input, size_t len, const transit_read_options_t *opts); transit_write() Encode a value with the given codec, appending to a growable output buffer. int transit_write(const transit_codec_t *codec, transit_value_t v, transit_outbuf_t *out); Parameters:

codec: the target wire format v: the value to encode out: an initialized transit_outbuf_t; encoded bytes are appended to out->data (out->len bytes)

Returns: 0 on success, or a non-zero transit_error_t on failure. transit_write_opts() Like transit_write(), but with explicit options (verbose output, cache toggle). int transit_write_opts(const transit_codec_t *codec, transit_value_t v, transit_outbuf_t *out, const transit_write_options_t *opts); transit_result_free() Free a result and the entire value tree it owns. void transit_result_free(transit_result_t *r); Note: this frees the backing arena in one shot. Do not free individual values, and do not use any borrowed spans afterwards. Codecs const transit_codec_t *transit_codec_json(void); /* compact JSON */ const transit_codec_t *transit_codec_json_verbose(void); /* verbose JSON */ const transit_codec_t *transit_codec_msgpack(void); /* MessagePack */ Each returns an immutable, process-global descriptor that is safe to share across threads.

Type System transit_value_t is an exposed, by-value tagged union — you can inspect it directly or via the accessor functions. Its kind is one of:

transit_kind_t Payload Notes

TRANSIT_NULL —

TRANSIT_BOOL bool

TRANSIT_INT int64_t signed 64-bit

TRANSIT_DOUBLE double

TRANSIT_STRING transit_span_t UTF-8 bytes

TRANSIT_BYTES transit_span_t raw bytes (base64 on the JSON wire)

TRANSIT_KEYWORD transit_span_t interned name, e.g. :foo

TRANSIT_SYMBOL transit_span_t

TRANSIT_URI transit_span_t

TRANSIT_UUID uint8_t[16] parsed 128-bit value

TRANSIT_INSTANT int64_t milliseconds since the Unix epoch

TRANSIT_CHAR int32_t Unicode codepoint

TRANSIT_BIGINT transit_span_t textual representation

TRANSIT_BIGDEC transit_span_t textual representation

TRANSIT_ARRAY items vector

TRANSIT_LIST items

TRANSIT_SET items

TRANSIT_MAP keys + vals preserves entry order

TRANSIT_TAGGED tag + representation extension point

A transit_span_t is a (pointer, length) view over bytes — not necessarily NUL-terminated: typedef struct { const uint8_t *ptr; size_t len; } transit_span_t; transit_kind_t transit_kind_of(transit_value_t v); /* the value's kind */ bool transit_is(transit_value_t v, transit_kind_t k); /* kind == k */ Scalar Accessors bool transit_as_bool(transit_value_t v); int64_t transit_as_int(transit_value_t