GitHub - jeffhajewski/latticedb: Embedded single-file knowledge graph database with vector search and full-text search for AI/RAG apps

Embedded property-graph database with native vector and full-text indexing. LatticeDB is a single-file local database for connected, semantic, and textual data. It lets you traverse relationships, run vector similarity search, and do BM25 full-text search over the same dataset in one engine and one query layer. It is designed for relationship-heavy workloads on a single machine, with zero-config operation and an embedded single-writer model. LatticeDB is an embedded, single-file graph database that lets local applications query the same data by relationship, semantics, and text, then consume durable graph and application events from the same file. Workloads like Graph RAG, agent memory, and local knowledge tools are examples built on those primitives, not the definition of the engine. One file. Your entire database is a single portable file. No server, no configuration. One query layer. Graph traversal, HNSW vector similarity, and BM25 full-text — in the same query language. One event log. Durable named streams and a built-in graph changefeed share the same transaction/WAL path as graph writes. Local-first. Designed for one owning process on one machine, with WAL-backed durability. Fast. 0.13 μs node lookups. 0.83 ms vector search at 1M vectors with 100% recall. -- Find chunks similar to a query, traverse to their document, then to the author MATCH (chunk:Chunk)-[:PART_OF]->(doc:Document)-[:AUTHORED_BY]->(author:Person) WHERE chunk.embedding <=> $query_vector < 0.3 AND doc.content @@ "neural networks" RETURN doc.title, chunk.text, author.name ORDER BY chunk.embedding <=> $query_vector LIMIT 10 Install CLI curl -fsSL https://raw.githubusercontent.com/jeffhajewski/latticedb/main/dist/install.sh | bash Python pip install latticedb Published wheels are expected to bundle liblattice on supported platforms. Source installs can also bundle a staged native library during wheel builds with LATTICE_BUNDLE_LIB_DIR=/path/to/lib. TypeScript / Node.js npm install @hajewski/latticedb Published package tarballs are expected to bundle liblattice on supported platforms. Source checkouts can stage the native library into the package with LATTICE_BUNDLE_LIB_DIR=/path/to/lib npm run bundle:native. Go See bindings/go/README.md for the current cgo workflow. The default consumer path uses installed pkg-config metadata; in-repo development can use -tags repolocal against zig-out/lib. There is also a runnable graph/vector/text retrieval example in examples/go. Recent binding-surface cleanups moved embedding helpers into dedicated modules and subpackages. See docs/client_api_migration.md for the preferred imports and current compatibility aliases. Start Here Getting Started maps the shortest path for CLI, Python, TypeScript, and Go. CLI Quickstart is the smallest copy-paste example in the repo. Examples Overview covers the larger graph/vector/text retrieval demos. Example A complete example: create a small knowledge graph with documents and authors, store embeddings, index text, then query across all three search modes. Python from latticedb import Database from latticedb.embedding import hash_embed with Database("knowledge.db", create=True, enable_vectors=True, vector_dimensions=128) as db: # --- Build the graph --- with db.write() as txn: # Create authors alice = txn.create_node(labels=["Person"], properties={"name": "Alice", "field": "ML"}) bob = txn.create_node(labels=["Person"], properties={"name": "Bob", "field": "Systems"}) txn.create_edge(alice.id, bob.id, "COLLABORATES_WITH") # Create documents with chunks for title, text, author in [ ("Attention Is All You Need", "The transformer architecture uses self-attention...", alice), ("Scaling Laws for LLMs", "We find that model performance scales predictably...", alice), ("Log-Structured Merge Trees", "LSM trees optimize write-heavy workloads...", bob), ]: doc = txn.create_node(labels=["Document"], properties={"title": title}) chunk = txn.create_node(labels=["Chunk"], properties={"text": text}) # Store embedding and index text txn.set_vector(chunk.id, "embedding", hash_embed(text, dimensions=128)) txn.fts_index(chunk.id, text) txn.create_edge(chunk.id, doc.id, "PART_OF") txn.create_edge(doc.id, author.id, "AUTHORED_BY") txn.commit() # --- Query: vector search + text match + graph traversal --- results = db.query(""" MATCH (chunk:Chunk)-[:PART_OF]->(doc:Document)-[:AUTHORED_BY]->(author:Person) WHERE chunk.embedding <=> $query < 0.5 RETURN doc.title, chunk.text, author.name ORDER BY chunk.embedding <=> $query LIMIT 5 """, parameters={"query": hash_embed("transformer attention mechanism", dimensions=128)}) for row in results: print(f"{row['doc.title']} by {row['author.name']}") # --- Full-text search --- for r in db.fts_search("self-attention transformer"): print(f"Node {r.node_id}: score={r.score:.4f}") # --- Aggregations --- stats = db.query(""" MATCH (doc:Document)-[:AUTHORED_BY]->(p:Person) RETURN p.name, count(doc) AS papers ORDER BY papers DESC """) for row in stats: print(f"{row['p.name']}: {row['papers']} papers") TypeScript import { Database } from "@hajewski/latticedb"; import { hashEmbed } from "@hajewski/latticedb/embedding"; const db = new Database("knowledge.db", { create: true, enableVectors: true, vectorDimensions: 128, }); await db.open(); // Build a graph await db.write(async (txn) => { const alice = await txn.createNode({ labels: ["Person"], properties: { name: "Alice", field: "ML" }, }); const doc = await txn.createNode({ labels: ["Document"], properties: { title: "Attention Is All You Need" }, }); const chunk = await txn.createNode({ labels: ["Chunk"], properties: { text: "The transformer architecture uses self-attention..." }, }); await txn.setVector(chunk.id, "embedding", hashEmbed("transformer self-attention", 128)); await txn.ftsIndex(chunk.id, "The transformer architecture uses self-attention..."); await txn.createEdge(chunk.id, doc.id, "PART_OF"); await txn.createEdge(doc.id, alice.id, "AUTHORED_BY"); }); // Query across vector search + graph traversal const results = await db.query( `MATCH (chunk:Chunk)-[:PART_OF]->(doc:Document)-[:AUTHORED_BY]->(author:Person) WHERE chunk.embedding <=> $query < 0.5 RETURN doc.title, chunk.text, author.name ORDER BY chunk.embedding <=> $query LIMIT 5`, { query: hashEmbed("attention mechanism", 128) } ); for (const row of results.rows) { console.log(`${row["doc.title"]} by ${row["author.name"]}`); } await db.close(); Go db, err := latticedb.Open("knowledge.db", latticedb.OpenOptions{ Create: true, EnableVectors: true, VectorDimensions: 128, }) if err != nil { log.Fatal(err) } defer db.Close() err = db.Update(func(tx *latticedb.Tx) error { node, err := tx.CreateNode(latticedb.CreateNodeOptions{ Labels: []string{"Chunk"}, Properties: map[string]latticedb.Value{"text": "The transformer architecture uses self-attention..."}, }) if err != nil { return err } if err := tx.SetVector(node.ID, "embedding", []float32{1, 0, 0, 0}); err != nil { return err } return tx.FTSIndex(node.ID, "The transformer architecture uses self-attention...") }) if err != nil { log.Fatal(err) } Performance Benchmarked on Apple M1, single-threaded, with auto-scaled buffer pool. Run zig build benchmark to reproduce. For the repeated-term FTS indexing workload that previously exposed quadratic append behavior, run zig build fts-benchmark. Core Operations Operation Latency Throughput Target Status Node lookup 0.13 μs 7.9M ops/sec < 1 μs PASS Node creation 0.65 μs 1.5M ops/sec — — Edge traversal 9 μs 111K ops/sec — — Full-text search (100 docs) 19 μs 53K ops/sec — — 10-NN vector search (1M vectors) 0.83 ms 1.2K ops/sec < 10 ms @ 1M PASS Vector Search (HNSW) at Scale 128-dimensional cosine vectors, M=16, ef_construction=200, ef_search=64, k=10. Run zig build vector-benchmark to reproduce. Scale Mean Latency P99 Latency Recall@10 Memory 1,000 65 μs 70 μs 100% 1 MB 10,000 174 μs 695 μs 99% 10 MB 100,000 438 μs 1.2 ms 99% 101 MB 1,000,000 832 μs 1.8 ms 100% 1,040 MB Search latency scales sub-linearly (O(log N)) with 99–100% recall@10. Uses heuristic neighbor selection (HNSW paper Algorithm 4) for diverse graph connectivity, connection page packing for ~4.5x memory reduction, and pre-normalized dot product for fast cosine distance. ef_search Sensitivity (1M vectors) ef_search Mean Latency Recall@10 16 506 μs 57% 32 1.9 ms 79% 64 990 μs 100% 128 3.2 ms 100% 256 11.6 ms 100% Competitive Analysis Point Lookups System Latency Type Source LatticeDB 0.13 μs Embedded zig build benchmark RocksDB (in-memory) 0.14 μs Embedded RocksDB wiki SQLite (in-memory) ~0.2 μs Embedded Turso blog SQLite (WAL, disk) 3 μs (p90) Embedded marending.dev Neo4j 28 ms (p99) Server Memgraph comparison LatticeDB's B+Tree achieves sub-microsecond cached lookups, matching RocksDB in-memory and outperforming SQLite on disk by 23x. Vector Search System Latency (10-NN) Scale Type Source LatticeDB 0.83 ms mean, 100% recall 1M Embedded zig build vector-benchmark FAISS HNSW (single-thread) 0.5–3 ms 1M Library FAISS wiki Weaviate 1.4 ms mean, 3.1 ms P99 1M Server Weaviate benchmarks Qdrant ~1–2 ms 1M Server Qdrant benchmarks Milvus + SQ8 2.2 ms P99 1M Server VectorDBBench pgvector HNSW ~5 ms @ 99% recall 1M Extension Jonathan Katz LanceDB 3–5 ms 1M Embedded LanceDB blog Chroma 4–5 ms mean 1M Embedded Chroma docs Pinecone P2 ~15 ms (incl. network) 1M Cloud Pinecone blog sqlite-vec (brute force) 17 ms 1M Extension Alex Garcia LatticeDB at 1M achieves 0.83 ms mean with 100% recall@10 — faster than FAISS single-threaded HNSW and competitive with Weaviate and Qdrant server-based systems (which add network overhead in practice). Graph Traversal System 2-hop (100K nodes) Type Source LatticeDB 39 μs Embedded zig build sqlite-benchmark SQLite (recursive CTE) 548 μs Embedded zig build sqlite-benchmark Kuzu 19 ms Embedded The Data Quarry Neo4j 10 ms (1M nodes) Server Neo4j blog LatticeDB vs SQLite — Social network graph with power-law degree distribution, adjacency cache pre-warmed: Small Scale (10K nodes, 50K edges)