Claude Code Rewrites PostHog's SQL Parser for 70x Speedup – How Property-Based Testing and Parallel Agents Worked
PostHog engineer Robbie Coomber used multiple long-running Claude Code sessions in parallel to rewrite their SQL parser, achieving a ~70x speedup over the existing ANTLR-generated C++ parser. The new parser is 16K lines of hand-rolled Rust code, plus 5K lines of tooling and tests.
Why rewrite?
PostHog transpiles user SQL to raw ClickHouse SQL for logical data views and optimizations. The parser turns SQL into an AST for downstream access control and optimization. The old ANTLR-generated parser used a generic graph-walking interpreter (an ATN — NFA-with-a-stack) with arbitrary dynamic lookahead, which was slow despite being in C++. Hand-rolled recursive-descent parsers are inherently faster.
Approach: parallel agent sessions + oracle TDD
- Tested two approaches in parallel: one focused on performance (recursive-descent with Pratt expression parsing), the other on correctness (mimicking ANTLR's behavior with explicit code). Both worked equally well.
- Used the existing C++ parser as an oracle to generate disagreements — find SQL where parsers differed, fix the new parser, repeat.
- Property-based testing generated countless SQL variations, including a test for
SELECT SELECT FROM FROM WHERE WHERE AND AND(valid SQL). - The new parser agrees with the oracle for all realistic queries; differences occur only for pathological queries.
Results
70x speedup in parsing. The final parser is a recursive-descent design with lookahead and backtracking only where necessary. Coomber notes that without AI, writing and maintaining such a hand-rolled parser would take months and likely not be worth the effort. With Claude Code, it became practical.
Key takeaway
This case study shows that parallel AI coding sessions, combined with property-based testing and an oracle for test-driven development, can dramatically improve performance-critical code. The technique — using agents to rewrite core infrastructure while relying on automated disagreement detection — is reusable for other projects.
📖 Read the full source: HN AI Agents
👀 See Also
Gemma 4 E2B Tested as Multi-Agent Coordinator in TypeScript Framework
A developer tested Gemma 4 E2B as a coordinator in a multi-agent setup using the open-multi-agent TypeScript framework. The model successfully decomposed tasks into JSON, assigned agents, called tools like bash and file operations, and synthesized results.
AVP Protocol Enables LLM Agents to Share KV-Cache Instead of Text for Token Efficiency
AVP (Agent Vector Protocol) allows LLM agents to pass KV-cache directly between them instead of text, reducing token processing by 73-78% and achieving 2-4x speedups across Qwen, Llama, and DeepSeek models. The protocol works with HuggingFace and vLLM connectors and is available as a Python package.
TestThread: Open Source Testing Framework for AI Agents
TestThread is an open source testing framework for AI agents that runs tests against live endpoints, provides pass/fail results with AI diagnosis, and includes features like semantic matching, PII detection, and CI/CD integration.
EvalShift: Open-source CLI for detecting LLM regressions during model migration
EvalShift is an MIT-licensed Python CLI that compares source vs target LLM outputs across prompts, agents, and tool-calling workflows, generating a local HTML regression report.