Optimizing Qwen 3.6 27B/35B on RTX 3090: Flags, Quantization, and Auto-Routing
A developer running Qwen 3.6 models locally on an RTX 3090 (24GB VRAM), Ryzen 5700X, 64GB RAM, Windows 11, is hitting performance and reliability issues. They're using llama-server with custom flags and seeking advice on quant choice, throughput, and automatic model routing.
Commands and Quantizations
35B (UD Q4_K_M):
llama-server.exe -m "path\Qwen3.6-35B-A3B-UD-Q4_K_M.gguf" -ngl 99 -c 131072 -np 2 -fa on -ctk f16 -ctv f16 -b 2048 -ub 512 -t 8 --mlock -rea on --reasoning-budget 2048 --reasoning-format deepseek --jinja --metrics --slots --port 8081 --host 0.0.0.027B (UD Q4_K_XL):
llama-server.exe -m "path\Qwen3.6-27B-UD-Q4_K_XL.gguf" -ngl 99 -c 196608 -np 1 -fa on -ctk q8_0 -ctv q8_0 -b 2048 -ub 512 -t 8 --no-mmap -rea on --reasoning-budget -1 --reasoning-format deepseek --jinja --metrics --slots --port 8081 --host 0.0.0.0Reported Issues
- 35B too slow – even simple iterative tasks feel unusable.
- 27B faster but unreliable – code output breaks; simple tasks can take 20–30 minutes.
- Manual model switching – must kill server, paste new command, reload model.
Specific Questions
- Are the flags suboptimal? (e.g., context size, batch size, cache type)
- Which quant / model gives best balance of speed and coding accuracy on 24GB VRAM?
- How to auto-switch models per request, or keep multiple models warm and route?
Context
The user runs Hermes agent on a Raspberry Pi 5 for scraping and automation, and local coding with OpenCode/QwenCode. They want a setup that doesn't require manual server restarts.
📖 Read the full source: r/LocalLLaMA
👀 See Also
System Architecture for Vibe Coders: A Senior Engineer's Guide
A 10-year engineer shares how to approach app building with Claude Code: start at the system level, not the code. Covers the four components — frontend, backend, database, plumbing — with a deep dive into the plumbing: APIs, hosting, deployment, secrets, and security.
Windows Cowork VM Service Error: Path Issue and Fix
A Windows Cowork installation issue causes the 'VM service not running' error every 10-20 minutes due to incorrect vm_bundles folder path in MSIX installs. The fix involves locating the correct folder and using a repair script.
Fine-Tuning Qwen 3:0.6B for Question Categorization – Baseline vs Finetuned Results
Fine-tuning a tiny 0.6B parameter LLM (Qwen 3:0.6B) with ~850 household questions using Unsloth. Baseline prompting scored 10% accuracy; finetuned results likely exceed 80-90%.
12 OpenClaw Power User Tips for Efficient AI Agent Workflows
A Reddit post outlines practical strategies for optimizing OpenClaw usage, including splitting conversations into topic-specific threads, using voice memos for input, matching models to tasks, delegating work to sub-agents, and implementing security layers.