Building a Jarvis-Style AI with MCP and Open Models
A developer on r/LocalLLaMA shared a Jarvis-style AI assistant built using the Model Context Protocol (MCP) and open models. The system is still work in progress but already supports several agentic features.
Key Details
- Memory — persistent context across sessions
- Autonomous file editing — the AI can read and write files on its own
- Visible tool-call tracing — users can see each step the AI takes
- Confirmation before dangerous actions — safety guardrails for destructive operations
- Persistent context — maintains state over time
- Hybrid local/cloud inference — can switch between local and cloud models
The primary model is Qwen3-Coder 480B running in the cloud for reasoning and tool usage, because smaller local models hallucinate too much with MCP workflows. The user is now working on Windows UI automation, dynamic skills, and AI-generated skill creation so the assistant can expand its own capabilities. The goal is a practical AI operator that interacts with and automates the user's computer, not just another chatbot.
Who It's For
Developers experimenting with local AI agents, MCP-based tool use, and autonomous computer automation using open-weight models.
📖 Read the full source: r/LocalLLaMA
👀 See Also
Local Memory System for AI Coding Tools Extracts 2,600+ Facts from Conversation Logs
A developer built a local memory layer that ingests conversation logs from Claude Code, Factory.ai, and Codex CLI, extracts structured facts using a local LLM, and auto-injects context into new sessions. After months of use, it has indexed 13,000+ messages and extracted 2,600+ facts.
Cascade Graph: Interactive Map of AI Energy Constraints and Supply Chokepoints
The Cascade Graph is a directed knowledge graph of the physical economy, mapping 405 nodes (drivers, chokepoints, geographies, tickers) connected by 597 sourced edges to trace how structural pressure cascades from energy and physics constraints through to investable assets.
Agents & A.I.mpires: Strategy Game Where AI Agents Play and Humans Spectate
Agents & A.I.mpires is a persistent real-time strategy game on a hex-grid globe where AI agents autonomously claim territory, attack, form alliances, and write daily war blogs via HTTP API calls. Humans only spectate the emergent behavior.
WCY format reduces LLM token overhead by 50-71% and adds structural 'I don't know' markers
WCY (Watch-Compute-Yield) is a line-oriented format that reduces JSON token overhead by 50-71% and introduces structural '?' markers for LLMs to indicate uncertainty during reasoning. The format requires no fine-tuning—just three few-shot examples.