boxBot: An Open-Source Smart Speaker Powered by Claude and Hailo AI
A developer known as FunScore645 built boxBot, a smart speaker that uses Claude for agent-driven hardware control instead of rigid rules. The project is open-sourced at github.com/dv-hart/boxbot and is still a work in progress but already useful for daily family tasks.
Hardware and Build
The hardware stack includes:
- Raspberry Pi (model not specified)
- Hailo AI accelerator (for edge inference, presumably)
- Respeaker microphone array
- Raspberry Pi camera
- Waveshare display
- Speakers
The enclosure is handcrafted from walnut, with passive cooling—the developer used Claude to calculate thermal loads and dissipation rates for the design.
Agent-Driven Architecture
Instead of complex conversation rules, the agent (likely Claude-based) controls the hardware directly through a custom Python SDK. Key capabilities:
- Dynamic muting: The agent decides when to mute the mic to ignore background noise or side conversations.
- Barge-in handling: Agent-driven logic manages interruptions without explicit rules.
- Multi-channel response: Can reply via voice or WhatsApp, chosen by the agent.
- Widget-based display: The screen uses a JSON widget framework—the agent reads and manipulates widgets via JSON without screenshots.
Skill System and Flexibility
The agent uses a small set of tools, with most logic routed through Python scripts and the boxBot SDK. Skills are defined in Markdown files. Example: when the developer added a family calendar event, his wife wanted a text notification. He added one line in a .md file, and the agent updated the calendar skill to send a WhatsApp message automatically on new events.
Memory and Security
The developer mentions a custom memory system and security considerations (camera+mic in a living room) but deferred details to comments. The agent is tightly coupled with hardware for security.
Who It's For
Developers interested in building AI-driven hardware projects with agent-controlled peripherals and open-source foundations.
📖 Read the full source: r/ClaudeAI
👀 See Also
AI Agent Embedded in Shell: Terminal Buffer & Overlay Extension
Open-source shell with embedded AI agent that reads terminal output and types commands via a floating overlay. Supports local and cloud models.
Apideck CLI: A Low-Context Alternative to MCP for AI Agents
Apideck CLI is an AI-agent interface that uses ~80 tokens for its agent prompt instead of tens of thousands for tool schemas, addressing MCP's context window consumption problem. Benchmarks show MCP can cost 4 to 32× more tokens than CLI for identical operations.
Multi-Agent Debate Approach Improves LLM Reasoning Quality
A developer experimented with a multi-agent debate approach using CyrcloAI, where different AI agents take on roles like analyst, critic, and synthesizer to critique each other's responses before producing a final answer, resulting in more structured and deliberate outputs.
Benchmark shows AI browser automation tools vary 2.6x in token costs despite identical accuracy
A benchmark of 4 CLI browser automation tools using Claude Sonnet 4.6 on 6 real-world tasks found all achieved 100% accuracy, but openbrowser-ai used 36,010 tokens while others used 77,123-94,130 tokens. Tool call count was the strongest predictor of token cost.