Giving Claude a Local LLM as an Assistant via MCP on Mac
A Reddit user detailed how they gave Claude access to a local LLM running on a Mac Mini M4 (24GB RAM) via an MCP connection to Ollama. The setup uses Ollama serving Qwen 2.5 Coder (14B) as an assistant named 'Frank', which Claude can delegate tasks to under specific rules — must use fewer tokens than Claude itself, must not affect quality, and requires a final review.
Setup Details
- Hardware: Mac Mini M4 with 24GB RAM.
- Local LLM: Qwen 2.5 Coder (14B) running via Ollama (also tested with LM Studio).
- Connection: MCP (Model Context Protocol) to link Claude (CLI or Desktop App) with the local model.
- Instructions: Claude was given a memory Markdown file (
memory.md) with guidelines for when and how to use Frank — e.g., delegate text processing, large CSS/HTML file handling, and use only when it saves tokens without degrading output quality.
Practical Use Cases
- Text processing and transformation — offloaded to Frank to reduce Claude's token usage.
- Handling large CSS/HTML files that would be expensive for Claude to process directly.
- Running performance, coding, and logic tests — Claude evaluated local models via Frank rather than manually.
The user noted they are operating at the limits of their RAM/GPU and cannot test larger models (30B+). They invited others with more powerful hardware to try similar setups and share results.
This approach effectively creates a cost-free assistant for Claude, offloading token-heavy tasks while maintaining quality through Claude's final review.
📖 Read the full source: r/ClaudeAI
👀 See Also
Crime Team: Multi-Agent Orchestrator for OpenClaw — Parallel Code Review with Coder Agent
Crime Team v0.1 runs multiple specialist OpenClaw agents in parallel for code review, then integrates findings. Includes per-agent models, a coder agent that applies changes, and a re-audit loop. CLI + GUI.
companion-capture: Tool saves Claude Code's ephemeral speech bubbles
companion-capture is an open-source tool that captures Claude Code's companion character speech bubbles before they vanish from the terminal. It saves messages to markdown files and SQLite for search, using VT100 screen buffer parsing to track cursor positions.
Sx: An Open-Source Package Manager for AI Skills, MCPs, and Commands
Sx is a private npm-like package manager for AI assets—skills, MCP configs, commands, hooks, and agents—that lets teams share, version, and scope AI configurations across any AI client (Claude Code, Cursor, Copilot, Gemini).
Skillware adds synthetic data generator with entropy scoring for local model fine-tuning
Skillware has released a new synthetic data generator skill that uses zlib compression-ratio heuristics to score output diversity, helping prevent model collapse. The tool works out-of-the-box with Ollama, supports Gemini/Anthropic for high-reasoning batches, and outputs JSON batches for .jsonl fine-tuning pipelines.