Lathoa: A Kids' Math App Where the AI Is Meant to Be Wrong
Lathoa is a math practice app for kids roughly 10 to 14 years old. A robot character named Errol walks through a math problem step by step, and one of those steps is deliberately wrong. The kid's job is to spot the bad step and explain what's wrong with it. Sometimes nothing is wrong — so just hitting "there's a mistake" every time won't work. There's a playable case on the homepage with no signup.
How scoring works
- If the user finds a real error, she has to enter an explanation to gain extra XP.
- Speed also factors into the score — faster answers earn more points.
- There is no direct interaction or chatting with an LLM; the kid never prompts a model.
The hard part: making the model wrong on purpose
The author calls out the counterintuitive finding: getting an LLM to be wrong on purpose is difficult. Roughly half the time the model gives the correct answer and labels it wrong, or produces a "mistake" that's actually correct. That forced a verification pipeline before any case reaches a kid.
Evaluation pipeline
- A plain arithmetic check redoes the math exactly where possible.
- A second model solves the same problem without seeing Errol's work. If the two models disagree, the case is thrown away.
- Lathoa's harness is described as stable, with many evaluation steps to catch inconsistencies and prompt injections.
- Known weak spot: the second model can make the same mistake as the first. The arithmetic check exists to catch that.
- That arithmetic check currently only works on English cases. German and Greek use a comma for decimals, and the parsing isn't right yet.
The open pedagogical question
The author's real ask is whether finding someone else's mistake teaches something that solving the problem yourself doesn't. He's not sure, and wants to hear from teachers. If you work with this age group, that's the thread to weigh in on.
Worth noting for anyone building similar tools: the two-model + symbolic-check pattern here is a reasonable template for any task where you need a verifiably specific output, not just a plausible one. The failure mode — a verifier model sharing the same blind spot as the generator — is the classic limitation you have to design around, and this project names it openly.
📖 Read the full source: HN LLM Tools
👀 See Also
Pi Coding Agent with Qwen 35B Q2: Using Filesystem as External Memory and Enforcing Context Guards
A Reddit user built a stack around Pi coding agent with Qwen 35B Q2_K_XL quant that enforces guards — rejects edits over 100 lines, caps thinking blocks at 2000 chars, and monitors context at 65%/80% — treating the filesystem as the model's memory, not the context window.
Soul MCP Server Adds Persistent Memory and Safety for Local LLMs
Soul is an open-source MCP server that provides persistent memory across sessions for local LLMs with two commands: n2_boot at start and n2_work_end at end. It includes Ark safety features that block dangerous commands like rm -rf and DROP DATABASE at zero token cost, plus cloud storage configuration.
ClamBot: AI Agent Runs LLM-Generated Code in WASM Sandbox for Security
ClamBot is an AI agent framework that executes all LLM-generated code in a WebAssembly sandbox using QuickJS in Wasmtime, eliminating the need for exec() or subprocess calls. It includes an approval gate for tool calls, persistent script caching as 'clams', and supports multiple LLM providers.
CloudRouter Empowers AI Coding Agents with VM and GPU Management
CloudRouter introduces a CLI tool that allows AI coding agents to autonomously spin up cloud VMs and GPUs, automating tasks like browser verification and GPU-intensive workloads.