Fixing OpenClaw's Blind Spots: Building a Sitemap to Fetch All Anthropic Blogs
A reader noticed that OpenClaw's summary of Anthropic's tech blogs was missing some posts — specifically one from April 8th. The root cause: Anthropic hosts their blogs at multiple distinct URLs, and OpenClaw's browser tool doesn't crawl all of them. The fix involved generating a sitemap by having the agent click every link to build a complete list, then feeding that sitemap into the summarization pipeline. The solution is packaged as a skill for easy reuse, though it does cost extra tokens to generate the sitemap due to the link-clicking process.
Key Takeaway
- Anthropic's blog URLs are not unified — they spread content across different hostnames, making naive scraping incomplete.
- OpenClaw's browser tool for RSS/blog aggregation doesn't auto-discover all pages; it needs a complete URL list upfront.
- Manually generating a sitemap by traversing all links works, but it consumes tokens proportional to the site's size.
This is a pragmatic workaround rather than a deep fix, but it's a useful pattern for any site with fragmented content. If you need reliable coverage of multiple-blog-host setups, pre-seeding with a sitemap is the simplest approach.
📖 Read the full source: r/openclaw
👀 See Also
Windows System Tray Monitor for Claude Code Quota
A Windows system tray application that monitors Claude Code usage with a color-coded icon, auto-refreshes quota data every 5 minutes via Anthropic's OAuth API, and provides detailed dashboards showing hourly, daily, weekly, and monthly usage patterns.
HF Viewer: Visualize Any Hugging Face Model Graph Instantly
HF Viewer is a browser-based tool that renders an interactive architecture graph for any Hugging Face model. Paste a URL or repo name, inspect the graph without local setup.
ClearSpec: A Spec Generator to Reduce Hallucination in Claude Code
ClearSpec is a tool that generates structured specifications from plain English descriptions, connecting to GitHub repos to reference real file paths and dependencies, then uses those specs as prompts for Claude Code to provide better context.
OpenClaw skill reduces accessibility tree tokens from 600K to 1.3K for ad-heavy sites
A developer built an OpenClaw skill that uses ML-based element ranking to prune accessibility trees, cutting slickdeals.com from ~598K tokens to ~1.3K tokens by keeping only the top ~50 actionable elements.