A/B Test Results: oh-my-claudecode Hooks Show Minimal Impact on Claude Code Performance

Experiment Setup and Methodology

A developer conducted a straightforward A/B test to evaluate the impact of oh-my-claudecode dynamic hooks on Claude Code performance. The experiment used Claude Sonnet 4.6 with the same coding task run six times total: three runs with OMC hooks ON and three runs with OMC hooks OFF. All runs used identical prompts, session flow, and machine environment.

The task involved building a Markdown editor from scratch in a single continued session, implementing the app, writing and running tests, then reviewing and fixing bugs, security, and accessibility issues before running tests again. The goal wasn't to produce flawless code but to measure whether hooks changed Claude's behavior in meaningful ways.

Key Results

• Code quality: Identical scores (15.0/20 for both OFF and ON)
• Total cost: Literally the same ($5.56 for both conditions)
• Total tokens: Slightly lower with hooks ON (6.48M vs 6.76M OFF)
• Wall time: Hooks ON was faster on average (1,673s vs 2,152s OFF)
• Tool calls: Slightly fewer with hooks ON (37.0 vs 40.7 OFF)

Notable Observations

The SessionStart hook significantly increased cache read tokens on the first prompt: 424K with hooks ON versus 195K with hooks OFF (a 117% increase). However, this extra context didn't translate to better output for this particular task. Interestingly, by Prompt 3, hooks OFF actually used more cache reads than hooks ON.

The experiment's repository is available at https://github.com/ej31/omc-hook-experiment.

Developer Takeaways

The developer concluded that dynamic hooks are not a "magic better-code button" for single-session coding tasks, as Claude already performs well with static instructions. Any hook-related gains appear small enough to be drowned out by normal run-to-run variance. The bigger constant overhead might actually come from OMC's static footprint—MCP tool definitions, skills, agent catalog—rather than just dynamic hook behavior.

Importantly, this experiment only tested hook behavior, not whether the broader OMC ecosystem is helpful in larger, multi-session workflows. The developer plans to test further by removing oh-my-claudecode entirely and having Claude build a Snake game from scratch.