oMLX introduces SSD KV caching for Apple Silicon, reducing OpenClaw response times from 30-90 seconds to 5 seconds
What oMLX solves
Running OpenClaw locally typically means sending the same massive system prompt (20-30k tokens covering tools, skills, workspace context) on every request. While Ollama and LM Studio cache KV state, they invalidate the entire cache and recompute from scratch when context shifts mid-session, resulting in 30-90 second response times.
oMLX fixes this by persisting KV cache blocks to SSD in safetensors format. When a previously seen prefix returns, it's restored from disk instead of recomputed - working across requests and server restarts. Since OpenClaw's system prompt is mostly static (only timestamps and runtime metadata shift), SSD caching means only changed parts get recomputed.
Performance benchmarks
Tested with Qwen3.5-122B-A10B-4bit on M3 Ultra 512GB:
- Single request benchmarks:
- 1k context: 768 tok/s prompt processing, 56.6 tok/s generation, 65.5 GB peak memory
- 8k context: 940 tok/s prompt processing, 51.4 tok/s generation, 69.3 GB peak memory
- 32k context: 764 tok/s prompt processing, 42.4 tok/s generation, 73.4 GB peak memory
- Continuous batching (pp1024/tg128):
- 1x batch: 56.6 tok/s, 1.00x speedup
- 2x batch: 92.1 tok/s, 1.63x speedup
- 4x batch: 135.1 tok/s, 2.39x speedup
- 8x batch: 190.2 tok/s, 3.36x speedup
Setup with OpenClaw
- Download the DMG from releases and drag to Applications
- Point it at your model directory (reuses LM Studio models, no re-download needed)
- Add oMLX as a custom provider in openclaw.json
- The web dashboard generates the exact config - no terminal needed
Additional features
- Multi-model serving: LLM + embedding + reranker simultaneously
- Tool calling for all major formats (JSON, Qwen, Gemma, GLM) + MCP
- Tool result trimming - truncates oversized tool outputs
- OpenAI + Anthropic /v1/messages drop-in compatibility
- Native macOS menu bar app (not Electron)
- Apache 2.0 license, 100% open source
📖 Read the full source: r/openclaw
👀 See Also
Keep My Claw: Backup Service for OpenClaw Workspaces
Keep My Claw is a backup service that encrypts OpenClaw workspace data locally before uploading to Cloudflare R2. It handles memory files, cron jobs, skills, credentials, and config snapshots with scheduled backups and one-command restores.
Reasoning Guard: Proxy-Level Loop Detection for Local LLM Inference
A proxy-layer guard that detects and recovers from LLM reasoning loops using deterministic stream checks — token caps, n-gram repetition, and sentence fingerprinting — without model modifications.
Agent Skill Harbor: GitHub-native skill management for AI agent teams
Agent Skill Harbor is an open-source platform for teams to share, track, and govern AI agent skills using GitHub-native workflows. It collects skills from GitHub repos, tracks provenance, supports safety checks, and publishes a static catalog site with GitHub Actions and Pages.
Anthropic Open-Sources Claude for Legal: Plugin Suite for Contract Review, NDA Triage, and More
Anthropic released Claude for Legal, a repo of plugins, agents, and MCP connectors for legal workflows including vendor agreement review, NDA triage, and regulatory monitoring.