Analyzing 7 Years of Diary Entries with an LLM: RAG vs Fine-Tuning Failures

A developer on r/ClaudeAI shared their experience feeding 200+ personal diary entries (spanning 2019–2026) to an LLM for longitudinal analysis. The goal: detect behavioral patterns and measure how they changed over 7 years. The technical path was full of dead ends.

Key Technical Failures

• RAG (Retrieval-Augmented Generation) failed — the diary entries were too similar, causing retrieval to return semantically overlapping chunks. The model couldn't produce coherent longitudinal insights.
• Fine-tuning failed — due to the small dataset (200 entries), the model overfit and couldn't generalize patterns across time.
• Privacy constraints — using cloud APIs was not an option; the author needed local processing to keep sensitive diary data secure.

The Workaround

The final approach involved chunking entries by year, summarizing each year with a local LLM (likely Llama or Mistral via Ollama), then feeding the seven year-summaries back into the model for cross-year analysis. This hierarchical summarization bypassed RAG's limitations and avoided the need for large-scale fine-tuning.

Surprising Insight

The LLM identified a recurring pattern: the author rediscovers the same life lessons approximately every two years, as if encountering them for the first time. This suggests that insight without an enforcement mechanism doesn't stick — a meta-lesson about human behavior and LLM-assisted reflection.

Who This Is For

Developers working on personal analytics projects, privacy-preserving LLM pipelines, or longitudinal text analysis with small datasets.

The author published a full write-up with five insights and implementation details at the link below.