Multi-provider LLM fallback chain with Ollama support in production AI IDE
Resonant Genesis, a production AI IDE platform, has integrated local LLM support as a first-class provider in its architecture. The platform runs across 30+ microservices and treats local models as equal to cloud providers like Groq, OpenAI, Anthropic, and Gemini.
Architecture and integration
The platform uses a shared rg_llm library called UnifiedLLMClient that's volume-mounted across all services. Every microservice that needs LLM capabilities imports this same client. The fallback chain is configured as: Groq → OpenAI → Anthropic → Gemini → Ollama/LM Studio.
The IDE's thin client extension automatically discovers local Ollama models and adds them to the provider list. Users can configure the system to prefer local models first if desired.
Server-side orchestration
All orchestration lives server-side, with the IDE acting as a thin client that renders UI, executes local tools (file operations, terminal, git), and streams results via Server-Sent Events (SSE). The agentic loop, tool selection, system prompts, and LLM routing all happen on the server.
When using a local model, it still goes through the same governed execution pipeline:
- Pre-execution policy enforcement (blocks actions before they run)
- Native function calling via provider APIs (no JSON prompt injection)
- Cryptographic identity (DSID on Ethereum L2) for every agent
- Same 59 local tools available regardless of which LLM provider you choose
Benefits for local LLM users
For users running Ollama locally, this architecture provides:
- Privacy: Thin client architecture means no company intelligence in the binary, and with local models, prompts stay local
- Tool use: 59 local tools with native function calling, not prompt-injected JSON schemas
- Fallback: If a local model can't handle a complex task, it automatically falls back to cloud providers
The developers are seeking feedback from people running local models, particularly around function calling performance with smaller models and which models work well for agentic tool use.
The project is open source at GitHub, and a guest chat demonstrating the tool ecosystem is live at dev-swat.com (uses cloud models).
📖 Read the full source: r/LocalLLaMA
👀 See Also
Leanstral: Open-Source Code Agent for Lean 4 and Formal Proof Engineering
Mistral AI released Leanstral, the first open-source code agent designed for Lean 4, with 6B active parameters and Apache 2.0 licensing. Benchmarks show it outperforms larger open-source models and offers competitive performance to Claude at significantly lower cost.
Karpathy's autoresearch project: AI agents run overnight LLM training experiments
Andrej Karpathy released a minimal autoresearch project where an AI agent edits train.py, runs 5-minute nanochat training experiments, checks if val_bpb improved, and repeats overnight on a single GPU.
Open source AI model stack for cost-effective Claude replacement
A Reddit user shares a working AI model stack using open source models like Llama 3.3 70b and DeepSeek R1 32b for local execution, reducing monthly AI costs from £60+ to under £3 by routing 90% of tasks to free models.
Buyer Eval: Claude skill for B2B vendor evaluation using AI agent conversations
A Claude skill that evaluates B2B software vendors by researching your company, asking domain-specific questions, and directly interrogating vendor AI agents through the Salespeak Frontdoor API. It cross-references claims against independent sources and produces evidence-based scorecards with transparent verification levels.