agentmemory V4 achieves 96.2% on LongMemEval benchmark, outperforms commercial AI memory systems
agentmemory V4 is an open-source memory system for AI agents that just achieved a world record score of 96.2% on LongMemEval, the standard benchmark for long-term AI agent memory.
Benchmark Performance
The system outperformed several funded AI memory companies:
- PwC Chronos: 95.6%
- Mastra: 94.87%
- OMEGA: 93.2% (raw)
- Supermemory: 85.86%
- Emergence AI: 86%
- Zep: 71.2%
Development Details
Built solo in 16 days on a mid-range gaming PC (i3-12100F) with a total cost of $1,000. The system uses Claude Opus as a generator and GPT-4o as a judge, but the retrieval architecture is the core innovation.
Technical Architecture
The system combines multiple retrieval techniques in a single SQLite-backed system:
- HNSW (Hierarchical Navigable Small World) for approximate nearest neighbor search
- BM25 for traditional text retrieval
- Cross-encoder for relevance scoring
- Knowledge graph integration
- Temporal grounding for time-aware memory retrieval
Availability
The system is open source under the MIT license and available at: github.com/JordanMcCann/agentmemory
📖 Read the full source: r/LocalLLaMA
👀 See Also
Agent Telephony Layer Goes Open Source: Give Your AI a Phone Number
A dev built an open-source telephony layer for AI agents, enabling them to call and text. It's REST + MCP, framework-agnostic, and has an OpenClaw skill.
Prefex: A Local Proxy for Claude Code That Automates Prompt Caching and Session Memory
Prefex is a local proxy that sits between Claude Code and Anthropic's API, automatically injecting the header required for Anthropic's beta prompt caching feature. It also implements session memory to avoid resending full conversation history and includes a model router for cost optimization.
Docent: An AI Assistant for Paper Analysis Built with Claude Code
A developer created Docent, an AI assistant that reads uploaded papers, presents them, answers questions, and assesses understanding using Claude Code. The project is available on GitHub under MIT License with a demo on Vercel.
Needle: A 26M Parameter Tool-Calling Model Built Entirely Without FFNs
Needle is a 26M parameter function-calling model with no MLPs, achieving 6000 tok/s prefill and 1200 tok/s decode on consumer devices. It beats FunctionGemma-270M, Qwen-0.6B, Granite-350M, and LFM2.5-350M on single-shot tool calling.