Local AI Development with Qwen3.6-27B and Opencode on a 5090
A developer who previously dismissed local LLMs as 'not up to standards' compared to cloud offerings like Claude Code or Cursor recently switched to a fully local setup. Using Opencode + llama-server + Qwen3.6-27B at a reasonable quantization with 128K context, running on a single RTX 5090 in a dedicated Linux box. The setup serves over the network to their main dev machine.
Key Details
- Tooling: Opencode (frontend) + llama-server (backend) + Qwen3.6-27B model
- Hardware: 1× RTX 5090, dedicated Linux machine
- Context length: 128K tokens (user unsure if it can be pushed further, but found it sufficient)
- Performance: Not perfect — occasional loops require manual interruption — but overall 'very worthwhile'
Motivation
The switch was driven by increasing usage constraints and 'enshittification' of cloud plans. Local setup eliminates worries about usage limits, prompt analysis, or account bans — particularly important for security research, scraping, or other activities that might trigger cloud provider scrutiny.
Who It's For
Developers on the fence about local AI coding agents, especially those who have been skeptical about local model quality or who need to avoid cloud account risks. If you have a powerful GPU (e.g., RTX 5090), the experience is now competitive with cloud tools.
Bottom Line
The user reports 'immensely freeing' experience despite occasional hiccups, and believes local AI development has reached the point where it's 'very worthwhile indeed.'
📖 Read the full source: r/LocalLLaMA
👀 See Also
agent-recall: Local SQLite MCP for Persistent Claude Code Memory
agent-recall is an MCP server that gives Claude Code persistent memory across sessions using a local SQLite file. It provides 9 MCP tools for saving entities, relationships, and observations, with LLM-summarized briefings at session start instead of raw data dumps.
ProofShot: CLI for AI Agents to Verify UI Code with Browser Recording
ProofShot is a CLI tool that lets AI coding agents open a browser, interact with pages, record sessions, and collect errors, then bundles everything into a self-contained HTML file for review. It works with any AI agent via shell commands and is packaged as a skill.
Monarch v3: NES-Inspired KV Paging for 78% Faster LLM Inference
Monarch v3 implements NES-inspired memory paging for transformers, achieving 78% faster inference (17.01 to 30.42 tok/sec) on a 1.1B parameter model with nearly zero VRAM overhead. The open-source algorithm splits KV cache into hot and cold regions with compression and promotion mechanisms.
Claude Code HUD: Terminal Dashboard for Monitoring AI Coding Sessions
claude-code-hud is a terminal dashboard that provides real-time monitoring for Claude Code sessions, showing context window usage, API rate limits, and file changes without requiring an IDE. Run it with npx claude-code-hud.