ANE Optimization Through Phone-Steered AI Experiments Shows Kernel Fusion Benefits
A developer conducted 55 optimization experiments on the autoresearch-ane fork, primarily steering the process from their phone on a Saturday. The work focused on Apple Neural Engine (ANE) performance improvements through kernel optimization and architectural changes.
Performance Improvements
The experiments yielded measurable gains across several metrics:
- Validation loss decreased from 3.75 (a throwback from optimized 3.2) to 2.49
- Step time improved from 176ms to 96ms
- ANE utilization increased from 3.6% to 6.5%
Key Technical Change
The most significant improvement came from kernel fusion: "Fusing 3 ANE kernels into 1 mega-kernel eliminated 12 IOSurface round-trips per step - that single change beat every hyperparameter tweak combined." This architectural optimization proved more impactful than parameter adjustments.
Workflow Details
The developer used an unconventional approach:
- Ran experiments remotely, steering from their phone in brief moments
- Used Claude for brainstorming and pulling insights from public sources listed in the repository README
- Approached the problem with "short attention and minimal token input" - speculating on directions rather than dictating precise steps
- Completed 55 experiments with "several cases of actual typing"
- Worked in non-destructive mode only due to permission constraints ("no rm -rf /* and such")
Main Learning
Beyond the technical improvements, the developer noted: "Main learning isn't the improvement itself. It's that short attention and minimal token input - brainstorming direction, not dictating steps - can produce real measurable gains on a hard systems problem."
The work was conducted on the developer's laptop, and they mention an acceptance rate discrepancy: "55vs45 not quite mathing" in reference to experiment outcomes.
📖 Read the full source: r/LocalLLaMA
👀 See Also
PRECC Tool Cuts Claude Code API Costs with Pre-Tool-Call Compression
A developer built PRECC, an open source tool that intercepts Claude Code tool calls and compresses payloads using RTK (Redundancy-aware Token Kompression), reducing input tokens by 40-66% with no perceptible latency impact.
Routerly: Self-Hosted LLM Gateway with Runtime Routing Policies and Budget Control
Routerly is a free, open-source, self-hosted LLM gateway that provides runtime model selection based on routing policies like cheapest, fastest, or most capable, along with project-level budget limits with per-token tracking. It's OpenAI-compatible for drop-in use with tools like Cursor, LangChain, and Open WebUI.
Tendr Skill Adds CLI-Based Long-Term Memory with Hierarchy to Reduce Token Usage
A new OpenClaw skill separates reasoning from execution for long-term memory operations, using a CLI tool to handle structural changes deterministically. It supports wikilinks and explicit semantic hierarchy across files to reduce token consumption and prevent error accumulation.
Engram: A Learning Plugin for OpenClaw That Actually Makes Knowledge Stick
Engram is an OpenClaw plugin that uses neuroscience-based techniques like pre-testing, blind grading, and FSRS-4.5 scheduling to ensure you retain what you learn, even as AI agents accelerate your shipping.