Practical experience replacing automation stack with MCP servers and local LLMs
Setup and hardware
The developer runs a mix of Qwen 2.5 32B (quantized) and Llama 3.3 70B on a dual 3090 rig. Each automation task gets its own MCP server that exposes tools the model can call, functioning like an API that an LLM consumes instead of a human.
What works well
- Code review automation: Pointing the model at a git diff via MCP tools catches real issues including logic bugs, missing error handling, and race conditions. Works about 70% as good as a senior dev review.
- Log analysis and alerting: MCP server connects to ELK stack, with the model monitoring for anomaly patterns. It has caught 3 production issues before Grafana alerts fired. The key is giving enough context about what "normal" looks like for your system.
- Documentation generation: Model reads the codebase through MCP file tools and generates/updates API docs, saving hours per week with genuinely good output quality.
What doesn't work (yet)
- Multi-step reasoning chains: Anything requiring more than 3-4 tool calls in sequence starts to go off the rails as the model loses context of the original goal. Smaller context windows make this worse. Chain-of-thought prompting helps but doesn't solve it.
- Real-time decision making: Latency on 70B models means this can't be used for time-sensitive tasks. Code review pipeline takes 2-3 minutes per PR, making it fine for async workflows but useless for real-time applications.
- Creative problem solving: Local models struggle with tasks requiring approaches not well-represented in training data. API models (Claude, GPT-4) are noticeably better here.
Key architectural lessons
- Keep MCP servers stateless. Let the model manage state through tool calls, not server-side session.
- Build retry logic into your MCP client, not the server. Models will make malformed tool calls approximately 5% of the time.
- Log every tool call and response for debugging when the model does something unexpected.
- Use structured output (JSON mode) for anything downstream systems consume. Free-form text output is a debugging nightmare.
📖 Read the full source: r/LocalLLaMA
👀 See Also
Non-Coder Builds Multiplayer Game on Steam Using Claude AI — 60k Lines, 5 Factions, 87 Abilities
A Reddit user with zero coding experience built a full multiplayer game (60k lines, 5 factions, 87 abilities) using Claude AI and got it approved on Steam. Early Access June 1.
Claude + MCP Builds Running Routes, Syncs to Garmin Watch
A Reddit user hooked Claude up to a route builder via MCP and Garmin API. Claude finds flatter routes in hilly SF that the runner missed for a year. Routes are viewable, adjustable, and push directly to the watch.
LLMs generate SQL queries to analyze terabytes of CI logs in seconds
Mendral's AI agent traced a flaky test to a dependency bump three weeks prior by writing its own SQL queries, scanning hundreds of millions of log lines across a dozen queries in seconds. The system handles 1.5 billion CI log lines weekly, compressed 35:1 in ClickHouse.
Porting Linux to FPGA Soft Cores Using Claude Code
A developer ported and booted a nommu Linux kernel (v6.6.83) on the NEORV32 soft core using an FPGA setup with specific hardware configurations and open-source patches.