Local Voice Control Setup for AI Agents on Apple Silicon
This setup details how to implement local voice control for AI agents using Parakeet STT and Kokoro TTS on Apple Silicon, specifically tested on a Mac Mini M4. The goal was to achieve a fully local and fast voice interaction layer, eliminating dependencies on cloud services.
Key Details
- Hardware: Mac Mini M4 running OpenClaw + Claude as the AI agent.
- Software Setup: Parakeet for speech-to-text (STT) which transcribes voice input in approximately 240ms, and Kokoro for text-to-speech (TTS) that provides nearly instant responses.
- Benefits: Transitioning from typing to voice commands significantly enhances workflow flexibility, allowing for office-independent operation, such as from the balcony or while walking a dog.
- Challenges: Occasionally, the STT struggles with accent recognition, humorously leading to the AI agent correcting the user’s pronunciation.
- Enhancements: A browser extension incorporating a 3D avatar named Mimora enables visual interaction, showing various expressions like listening, thinking, and happy states during agent responses.
This configuration is ideal for those seeking cloud-independent, fast voice interaction with AI agents, particularly using Apple Silicon hardware.
📖 Read the full source: r/LocalLLaMA
👀 See Also
OpenLobster: Self-Hosted AI Agent in Go with 30MB RAM Footprint
OpenLobster is a self-hosted AI assistant written in Go that runs as a single binary with 30MB RAM usage and 200ms cold start. It supports multiple LLM providers including Ollama, OpenRouter, and any OpenAI-compatible endpoint, with memory stored in a graph database.
Holaboss AI Runtime Moves to TypeScript, Implements Persistent MCP Ports
The Holaboss AI local agent runtime has been refactored to use TypeScript exclusively, eliminating Python dependencies and reducing bundle size. It now persists MCP server ports in SQLite with UNIQUE(port) constraints to prevent collisions across restarts.
Lore: A tool that extracts structured context from AI coding conversations
Lore is a browser-based tool built with Claude Code that extracts structured context from AI conversations, capturing decisions, TODOs, blockers, and resume checklists. It's a React + TypeScript PWA with a Chrome extension for direct conversation capture and context injection.
Blackwell LLM Toolkit: NVFP4 Configs, Wheels, and Benchmarks for TensorRT-LLM on RTX Pro 6000
A community repo provides TensorRT-LLM configs, prebuilt LMCache wheels with sm_120 support, and benchmarks for Blackwell GPUs. Nemotron-3-Nano-Omni V3 hits 270 tok/s at 8k context on a single RTX Pro 6000.