Researchers Use Claude Projects for Academic Scoping Review: Strengths and Limitations
Claude as Research Assistant in Academic Review
Researchers conducted a peer-reviewed scoping review published in Artificial Intelligence in Education (Emerald, open access) using Claude Projects to analyze 39 qualitative interview studies from 20 countries on how students experience generative AI in higher education.
What Worked Well
- Cross-referencing themes between papers from structured spreadsheet data
- Augmenting human memory across a large dataset
- Suggesting analytical categories the researchers had not considered
- Acting as a "critical peer" for iterative thematic analysis
What Didn't Work Well
- Early CSV analysis was inaccurate and incomplete
- Prone to hallucination when outputs were not rigorously checked against the source spreadsheet
- Could be "lazy," not fully carrying out requests
- Sycophantic responses required explicit prompting for critique
- The learning curve meant it was not actually more efficient overall (productivity paradox)
Implementation Details
The researchers did not upload full papers due to copyright and ethical considerations. Instead, they uploaded their own structured notes into Claude Projects. Performance improved significantly when .xls support was added and again with Sonnet 3.7.
The researchers concluded that Claude was useful as a research assistant but required the same oversight you would give to a competent but unreliable colleague. Every output had to be verified against the original data. They plan to use it again, but only because they now understand its specific failure modes.
The paper is available open access under CC BY 4.0 license at https://doi.org/10.1108/AIIE-06-2025-0151.
📖 Read the full source: r/ClaudeAI
👀 See Also
BinktermPHP: A Full BBS/FidoNet Platform Built Primarily with Claude
BinktermPHP is an open-source web-based BBS written in PHP 8 with PostgreSQL, featuring a built-in binkp FidoNet mailer, echomail/netmail, file areas, door games, real-time server push, and an MCP server that exposes echomail to AI assistants. The developer reports building it almost entirely using Claude for implementation while handling architecture and testing.
Using Markdown Files as Workflow Engine for Claude Code in Kubernetes
A developer replaced traditional pipeline tools like Dagster or Prefect with plain English markdown files that Claude Code executes as a Kubernetes CronJob. The system coordinates steps by writing artifacts to disk and has been running for over a month.
Claude debugging case: Agent failed silently due to missing parameter, framing mattered more than model
A developer used Claude to build a calendar agent, then spent 40 minutes having Claude debug it before realizing the write_calendar tool lacked an attendees parameter. When given full context, Claude identified the issue in 10 seconds.
Autonomous Magazine Pipeline with Claude Code: Agentic Architecture Breakdown
A seven-step pipeline using Claude Code as an editorial team produces up to five fact-checked, multilingual articles per headline. The system includes five sub-agents, institutional memory via embeddings, and automated fact-checking against a growing database.