Fixing OpenClaw's Blind Spots: Building a Sitemap to Fetch All Anthropic Blogs
A reader noticed that OpenClaw's summary of Anthropic's tech blogs was missing some posts — specifically one from April 8th. The root cause: Anthropic hosts their blogs at multiple distinct URLs, and OpenClaw's browser tool doesn't crawl all of them. The fix involved generating a sitemap by having the agent click every link to build a complete list, then feeding that sitemap into the summarization pipeline. The solution is packaged as a skill for easy reuse, though it does cost extra tokens to generate the sitemap due to the link-clicking process.
Key Takeaway
- Anthropic's blog URLs are not unified — they spread content across different hostnames, making naive scraping incomplete.
- OpenClaw's browser tool for RSS/blog aggregation doesn't auto-discover all pages; it needs a complete URL list upfront.
- Manually generating a sitemap by traversing all links works, but it consumes tokens proportional to the site's size.
This is a pragmatic workaround rather than a deep fix, but it's a useful pattern for any site with fragmented content. If you need reliable coverage of multiple-blog-host setups, pre-seeding with a sitemap is the simplest approach.
📖 Read the full source: r/openclaw
👀 See Also
Local MCP Server Connects Claude to Mac Apps Without Cloud or Tokens
Local MCP is a native macOS MCP server that gives Claude Desktop, Cursor, Windsurf, and VS Code access to Mail, Calendar, Teams, and OneDrive data on your Mac without cloud processing or API tokens.
UK Sovereign LLM Inference: Relax.ai Launches Public Docs
Relax.ai released docs for UK sovereign LLM inference, redirecting to /docs/getting-started/introduction. The service was shared on HN with 104 points.
Spectyra Plugin for OpenClaw: Real-Time AI Cost Optimization by Analyzing Full Request Flow
Spectyra plugin reduces AI API costs by surfacing hidden waste like repeated calls, excessive context, and expensive model misuse in real time.
ToolLoop: Open-Source Framework for Claude-Style Tools with Any LLM
ToolLoop is an open-source Python framework with 11 tools for file operations, code search, shell access, and sub-agents that works with any LLM through LiteLLM. The 2,700-line framework allows switching models mid-conversation while maintaining shared context.