Building Jarvis: A Self-Hosted AI Operations Layer with OpenClaw

A Reddit user running a small architecture firm, construction companies, and a coworking space is building a personal AI assistant called Jarvis on a Mac mini using OpenClaw as the core agent framework. The goal: an always-on second brain that handles emails, deadlines, invoices, and client notes in the user's own tone.

Architecture Stack

• Agent Framework: OpenClaw (self-hosted)
• AI Model Cascade: Premium cloud model for complex tasks, lighter fallback, plus Qwen 2.5 7B via Ollama for fully local processing
• Workflow Automation: n8n
• Document Management: Paperless
• Structured Data: PostgreSQL
• Knowledge Base: Obsidian vault with a five-level hierarchy for easy navigation
• Infrastructure: Mac Mini (24/7), Git, Docker, Ollama, dedicated iCloud and Google accounts (Docs, Drive, Calendar), safety backups/Time Machine

Current Phase: Identity & Foundation

Jarvis is still in the foundation phase. The user is writing five identity notes that define personal decision-making, red lines, and communication style — before any agent actions happen autonomously. Next steps include:

• Vector search over the Obsidian vault
• A memory consolidation layer that builds daily/weekly summaries from conversations and logs

Planned Rollout

Low-risk tasks first: reading emails, alerting on deadlines. Gradually expand to drafting replies and processing invoices as trust builds. The first two weeks were spent configuring the Mac Mini (OpenClaw, models, remote access, Telegram, network hiding, passphrase, GPG). The user notes that Jarvis itself assisted in installing most applications last week.

Coming Next

Pending integration: LangGraph and additional tools. The user is asking the community for architectural advice, tool swaps, and pitfalls to watch out for.