Fine-Tuning Qwen 3:0.6B for Question Categorization – Baseline vs Finetuned Results

Torgeir Helgevold published a practical walkthrough of fine-tuning Qwen 3:0.6B to categorize household questions. The goal: narrow vector search space by mapping questions to categories like hvac, pool, and cooking before RAG retrieval.

Baseline Results – Prompting Without Fine-Tuning

Using the stock Qwen 3:0.6B model with a strict prompt ("Return only the category name from the list") yielded only 13 correct out of 131 test questions – 9.9% accuracy. Common failures: overusing broad labels like electric/appliances, inventing new categories (e.g., apartments), and returning null.

Fine-Tuning Setup

• Models used: Qwen 3:4B for general QA, Qwen 3:0.6B for classification
• Framework: Unsloth (open source, works with Qwen and Llama)
• Dataset: ~850 labeled entries – 70/15/15 split for train/eval/test
• Sample data:

  {
    "question": "When did we replace our pool pump?",
    "category": "pool"
  },
  {
    "question": "Who serviced the hot water heater for the home?",
    "category": "water heater"
  }

Key Takeaway

A 600M parameter model can be fine-tuned into a reliable classifier for a specific domain when given enough training data. The post suggests finetuned accuracy likely jumps from 10% to 80-90%+, making the tiny model suitable as a preprocessing step for RAG systems.