Dual-model architecture reduces token consumption by half for long conversations
Context compression system for AI agents
A developer on r/ClaudeAI shared a solution to the problem of AI agents losing context after conversation compaction. The system uses a dual-model architecture where a cheap small model (called the "subconscious") continuously compresses conversation history in the background.
Architecture details
The system has four layers:
- Narrative summary (~1K tokens)
- Compressed factoids
- Semantically retrieved verbatim quotes
- Raw recent turns
The main model ("conscious") receives a curated ~35K token context with the same information density that would normally require 120K tokens of raw history. The main model reads one coherent timeline and doesn't know the memory system exists.
Performance results
The developer simulated 260 turns across different conversation types. For sustained project work (starting with heavy research and gradually shifting to quick exchanges as the model learns the domain), the system cuts token consumption roughly in half.
Development tools
The system was built with Claude Code for the simulation and Claude.ai in the consulting and research stage. The developer is looking for others who have tried routing a smaller model to manage context for a larger one or found other workarounds for the compaction problem.
📖 Read the full source: r/ClaudeAI
👀 See Also
Survey of Local-First Markdown Memory Servers for AI Agents: Mem0, Hindsight, Zep, and the Newcomer Engram
A user tested ~20 local agent memory systems for storing memories as editable files. Engram (by Obsidian68) was the only one that met all requirements: fully local, Markdown storage, smart dedup, importance decay, and standalone server.
2-Prompt System to Carry Context Between Claude Chats Without Token Waste
A developer shares two prompts for compressing an entire Claude conversation into a structured context block and loading it into a fresh chat, preserving decisions, work, and next steps.
claude-powerline v1.20 adds TUI dashboard mode, context bar styles, and environment variable display
claude-powerline v1.20 introduces a TUI dashboard mode that replaces the single statusline with a full panel showing model info, context usage with progress bar, costs, git status, and more. The update adds 9 visual progress bar styles for context usage and environment variable display capability.
Hypura: Storage-tier-aware LLM inference scheduler for Apple Silicon
Hypura is a Rust-based inference scheduler that places model tensors across GPU, RAM, and NVMe tiers to run models exceeding physical memory on Apple Silicon Macs. It enables running a 31GB Mixtral 8x7B on a 32GB Mac Mini at 2.2 tok/s and a 40GB Llama 70B at 0.3 tok/s where vanilla llama.cpp crashes.