TinyFish Web Agent Outperforms Competitors in Web Task Benchmarking
The TinyFish Web Agent has proven to be a leading tool in tackling complex web tasks, achieving an 81.9% success rate on hard tasks in the Online-Mind2Web benchmark, which consists of 300 tasks across 136 live websites. This figure starkly contrasts with major competitors, such as OpenAI Operator, which managed only a 43.2% success rate on similar tasks.
The Online-Mind2Web benchmark is a rigorous measure of a web agent's capabilities, testing them on tasks ranging from easy, like browsing credit card offers on Marriott, to complex challenges such as booking event tickets with dynamic pricing. Tasks involve multiple steps with live websites, including handling form validation and pop-ups, making it a realistic test compared to other less reliable benchmarks like WebVoyager.
TinyFish distinguishes itself by handling compounding errors effectively. It drops only 15.6 points from easy to hard tasks compared to massive drops shown by other systems, highlighting its robustness in real-world scenarios. Notably, it has published all 300 task runs, including their 40 failures, which offers transparency into its performance characteristics and failure cases, such as infrastructure-level anti-bot blocks encountered on sites like apartments.com.
Developers looking for a robust web automation tool would find TinyFish's open-source cookbook repository of interest, which provides insight into its architecture and execution methodology.
📖 Read the full source: HN AI Agents
👀 See Also
UK Sovereign LLM Inference: Relax.ai Launches Public Docs
Relax.ai released docs for UK sovereign LLM inference, redirecting to /docs/getting-started/introduction. The service was shared on HN with 104 points.
Flash-MoE: Running 397B Parameter Qwen Model on MacBook Pro with Pure C/Metal
Flash-MoE is a pure C/Metal inference engine that runs Qwen3.5-397B-A17B, a 397 billion parameter Mixture-of-Experts model, on a MacBook Pro with 48GB RAM at 4.4+ tokens/second. The 209GB model streams from SSD through custom Metal compute shaders with no Python or frameworks.
Otterly: Route OpenClaw Through Your Claude Code Subscription
Otterly is a small npm package that exposes the local Claude CLI as an OpenAI-compatible HTTP server, letting you bill OpenClaw requests to your Claude Code subscription instead of pay-per-token API rates.
Vektori's Memory Architecture: Principles from Claude's Leaked System
Vektori implements a three-layer hierarchical sentence graph for AI memory, inspired by leaked principles from Claude's architecture. The system uses strict quality filters, skeptical retrieval with a 0.3 minimum score, and maintains correction history across sessions.