Code retrieval for AI agents: Why vector embeddings fail and per-file LLM graphs win
A year-long experiment building a code indexing system for AI coding tools yielded clear results: vector embeddings on code chunks and Tree-sitter AST parsing both have critical flaws, while per-file LLM analysis stored in a Neo4j graph with semantic fulltext search works best. The findings echo recent papers like RepoGraph (ICLR 2025) and Code-Craft.
Approaches tested
- Vector embeddings on code chunks – discarded entirely. A function named
process()in a payments service and one in an image pipeline embed to similar vectors, despite having nothing to do with each other. Vectors flatten call graphs, inheritance, imports — all structural relationships. Retrieval precision was unacceptable. - Tree-sitter AST parsing – precise and fast, but structural-only. It can tell you a function exists and what it calls, but cannot answer the question “this function handles webhook retries for failed Stripe payments.” Falls short when developers phrase questions in business language.
- Per-file LLM analysis → graph – works. Every file gets an LLM call generating
purpose,summary, andbusinessContext, stored as nodes in Neo4j with edges to classes, functions, keywords, and imports. Retrieval uses fulltext search across those semantic fields instead of vector similarity. SHA-256 diffing limits reindexing to changed files, making the upfront cost manageable.
Benchmarks from literature
RepoGraph (ICLR 2025) showed +32.8% improvement on SWE-bench with graph approaches. Code-Craft achieved +82% top-1 retrieval precision using bottom-up LLM summaries from code graphs.
Comparison to existing tools
The team published a side-by-side in comparison.md. Key differences:
- Bytebell: per-file LLM → purpose + summary + businessContext + entities; Neo4j + MongoDB storage; SHA-256 diff-aware reindex.
- PageIndex: TOC reasoning tree for long PDFs/docs; no code-specific semantics.
- GitNexus: Tree-sitter AST + community detection; optional per-symbol semantics; uses LadybugDB.
- GraphRAG: per-chunk LLM entities + community clustering for general text, not code.
- Sourcegraph/Cody: LSIF/SCIP search index; no per-node semantics; deployment is self-hosted or SaaS.
- Augment: proprietary semantic index with embeddings; SaaS-only; continuous indexing managed.
Open source
The system is open source at github.com/ByteBell/bytebell-oss.
📖 Read the full source: r/LocalLLaMA
👀 See Also
Femtobot: Efficient Rust Agent for Low-Resource Environments
Femtobot is a lightweight Rust-based AI agent designed to run efficiently on low-resource machines, such as older Raspberry Pis, through a ~10MB binary without large runtime dependencies.
OpenClaw Mock API for Building Tools/Integrations Against the Gateway
An open source mock of the OpenClaw gateway WebSocket API using Imposter. Supports chat.send/history, session create/list, agent list, model list. Extend with YAML and JSON.
Superglue CLI: Let AI Agents Execute API Calls Without Pre-Built Tools
Superglue CLI provides a skill that teaches AI coding agents how to use its commands, handle authentication, build tools, and debug failures. Instead of creating pre-built tools for every API integration, agents can read API specs at runtime and plan multi-step calls.
Bio-Inspired Memory System for Local LLMs: LTP and Selective Oblivion Implementation
A developer built a local MCP server implementing bio-inspired memory mechanics including Long-Term Potentiation reinforcement, selective oblivion decay, and weekly consolidation cycles. The system uses hybrid search with sqlite-vec and text fallbacks, non-blocking architecture with asyncio executors, and maintains state via a persistent 'Soul' file.