A/B Test Results: oh-my-claudecode Hooks Show Minimal Impact on Claude Code Performance
Experiment Setup and Methodology
A developer conducted a straightforward A/B test to evaluate the impact of oh-my-claudecode dynamic hooks on Claude Code performance. The experiment used Claude Sonnet 4.6 with the same coding task run six times total: three runs with OMC hooks ON and three runs with OMC hooks OFF. All runs used identical prompts, session flow, and machine environment.
The task involved building a Markdown editor from scratch in a single continued session, implementing the app, writing and running tests, then reviewing and fixing bugs, security, and accessibility issues before running tests again. The goal wasn't to produce flawless code but to measure whether hooks changed Claude's behavior in meaningful ways.
Key Results
- Code quality: Identical scores (15.0/20 for both OFF and ON)
- Total cost: Literally the same ($5.56 for both conditions)
- Total tokens: Slightly lower with hooks ON (6.48M vs 6.76M OFF)
- Wall time: Hooks ON was faster on average (1,673s vs 2,152s OFF)
- Tool calls: Slightly fewer with hooks ON (37.0 vs 40.7 OFF)
Notable Observations
The SessionStart hook significantly increased cache read tokens on the first prompt: 424K with hooks ON versus 195K with hooks OFF (a 117% increase). However, this extra context didn't translate to better output for this particular task. Interestingly, by Prompt 3, hooks OFF actually used more cache reads than hooks ON.
The experiment's repository is available at https://github.com/ej31/omc-hook-experiment.
Developer Takeaways
The developer concluded that dynamic hooks are not a "magic better-code button" for single-session coding tasks, as Claude already performs well with static instructions. Any hook-related gains appear small enough to be drowned out by normal run-to-run variance. The bigger constant overhead might actually come from OMC's static footprint—MCP tool definitions, skills, agent catalog—rather than just dynamic hook behavior.
Importantly, this experiment only tested hook behavior, not whether the broader OMC ecosystem is helpful in larger, multi-session workflows. The developer plans to test further by removing oh-my-claudecode entirely and having Claude build a Snake game from scratch.
📖 Read the full source: r/ClaudeAI
👀 See Also
Stanford Researchers Release OpenJarvis: A Local-First Framework for On-Device AI Agents
Stanford researchers have released OpenJarvis, a local-first framework for building on-device personal AI agents with tools, memory, and learning capabilities. The project includes GitHub repository and website links for developers to explore.
Silent Tool Failures in Coding Agents: A Hidden Efficiency Drain
Coding agents often encounter tool failures that go unnoticed because they fall back to alternative strategies, wasting tokens and reducing quality. The open-source tool Vibeyard detects these failures and suggests fixes.
Fullerenes: Open-source persistent memory layer for coding agents cuts tokens by 64% on SWE-bench
Fullerenes uses a local SQLite knowledge graph built via Tree-sitter to give coding agents like Claude Code persistent memory, reducing token usage by 64% on SWE-bench and up to 96.6% on internal benchmarks.
Open Memory Protocol: One Memory Store for Claude, ChatGPT, Cursor
Open Memory Protocol (OMP) is a vendor-neutral spec and reference server for portable AI memory. Run a self-hosted server via Docker or npx, connect Claude via MCP, and share memory automatically with ChatGPT and Cursor.