ReasonDB: Open-Source Document Database Using LLM-Guided Tree Navigation Instead of Vector Search

What ReasonDB Does

ReasonDB is an open-source document database that takes a different approach to knowledge retrieval by preserving document structure as a hierarchy (headings → sections → paragraphs) and letting the LLM navigate that tree to find answers. This addresses the common issue where retrieval failures in vector DBs and RAG pipelines become debugging black boxes.

How It Works

Ingest process: Documents are converted to markdown, chunked by structure, built into a tree, and each node is summarized by an LLM (bottom-up).

Query process: BM25 narrows candidate nodes → tree-grep filters by structure → LLM ranks by summaries → beam-search traversal over the tree extracts the answer. The LLM visits approximately 25 nodes out of millions instead of searching a flat vector index.

Query Language and Stack

ReasonDB uses RQL, an SQL-like query language:

SELECT * FROM contracts SEARCH 'payment terms' REASON 'What are the late payment penalties?' LIMIT 5;

Where SEARCH = BM25 and REASON = LLM-guided tree traversal.

The stack is built in Rust using redb, tantivy, axum, and tokio. It's a single binary that works with OpenAI, Anthropic, Gemini, Cohere, and compatible APIs (including local or OpenAI-compatible endpoints).

Practical Applications

This approach is particularly useful for developers who have been struggling with RAG retrieval quality or want to try structure-based retrieval instead of pure vector search. The tool was built after 3 years of experience with knowledge retrieval at Brainfish, where vector DBs, graph DBs, and custom RAG pipelines consistently presented the same debugging challenges.