10 AI Agents Tested on Qwen3.5-4B: Pi Finishes 5 Tasks in 1m36s, OpenCode Takes 17m43s
A developer benchmarked 10 AI coding agents on Qwen3.5-4B, each running headless with auto-confirmation and telemetry disabled where possible. Every agent got its own isolated settings folder, a fresh fixture copy, and 10 minutes per task. The full suite ran via adapters/<agent>.sh with automated checks from check/check.sh.
The 5 tasks
- T1 — Read a DB port from YAML (don't confuse it with the cache port), write it to a file.
- T2 — Write a
stats.pyscript, run it, save the output. - T3 — Fix a bug in
slugifyso tests pass; hidden examples catch overfitting. - T4 — Count ERROR lines across three log files, find the most frequent error.
- T5 — Rename a function across three files without changing program output.
Results (total time, tasks passed)
- Pi v1.3.0 — 1m36s, 5/5. T1 10s, T2 14s, T3 24s, T4 19s, T5 29s. Barely needs to reason about tool formats.
- goose v1.55.0 — 2m51s, 5/5. Consistent, no fumbling.
- Crush v0.98.1 — 3m31s, 5/5. Slightly more verbose reasoning.
- Atomic Agent v0.6.9 — 4m38s, 5/5. Slower on T3 (77s) and T5 (104s) due to multi-file edits, but no broken tool calls or false completions.
- Cline v3.0.73 — 5m56s, 5/5. Stalled over two minutes on T2 before writing correctly.
- OpenClaw v2026.10.5 — 5m44s, 5/5. Ran in
--local-model-leanmode; slower on T3 (107s) and T4 (116s). - OpenCode v1.19.3 — 17m43s. Completed everything but averaged nearly 4 minutes per task; the model overthought simple steps.
Seven agents are accounted for with times and pass counts. The source post lists 10, but only the top seven entries include benchmark tables in the excerpt. Raw per-task timings and the full adapter/check scripts are in the original thread.
Takeaway for anyone running small local models: normalization of tool-call formats matters more than raw model size here. Agents that hand the model a simple, rigid tool schema (Pi, goose) finished the same 5/5 in minutes, while more general-purpose loops burned time on formatting and overthinking.
📖 Read the full source: r/openclaw
👀 See Also
Atlassian Enables Default Data Collection for AI Training
Atlassian has enabled default data collection across its products to train AI models, according to a source posted on Hacker News with 312 points and 75 comments.
Anthropic launches Claude Community Ambassadors program
Anthropic has launched the Claude Community Ambassadors program, which provides resources for organizing local developer meetups and connecting builders worldwide. The program is open to participants from any background and location.
Claude Code System Prompt Assembly and Structure Revealed
A source map leak in Claude Code's npm package exposed the system prompt assembly flow, showing static prefix sections followed by dynamic session-specific content, with three identity variants and detailed execution guidelines.
CEOs Report Minimal AI Impact on Productivity and Employment in Recent Study
A study of 6,000 executives found 90% reported no AI impact on employment or productivity over three years, with average AI usage at 1.5 hours per week. Economists compare this to Solow's productivity paradox from the 1980s IT era.