Researchers Use Claude Projects for Academic Scoping Review: Strengths and Limitations
Claude as Research Assistant in Academic Review
Researchers conducted a peer-reviewed scoping review published in Artificial Intelligence in Education (Emerald, open access) using Claude Projects to analyze 39 qualitative interview studies from 20 countries on how students experience generative AI in higher education.
What Worked Well
- Cross-referencing themes between papers from structured spreadsheet data
- Augmenting human memory across a large dataset
- Suggesting analytical categories the researchers had not considered
- Acting as a "critical peer" for iterative thematic analysis
What Didn't Work Well
- Early CSV analysis was inaccurate and incomplete
- Prone to hallucination when outputs were not rigorously checked against the source spreadsheet
- Could be "lazy," not fully carrying out requests
- Sycophantic responses required explicit prompting for critique
- The learning curve meant it was not actually more efficient overall (productivity paradox)
Implementation Details
The researchers did not upload full papers due to copyright and ethical considerations. Instead, they uploaded their own structured notes into Claude Projects. Performance improved significantly when .xls support was added and again with Sonnet 3.7.
The researchers concluded that Claude was useful as a research assistant but required the same oversight you would give to a competent but unreliable colleague. Every output had to be verified against the original data. They plan to use it again, but only because they now understand its specific failure modes.
The paper is available open access under CC BY 4.0 license at https://doi.org/10.1108/AIIE-06-2025-0151.
📖 Read the full source: r/ClaudeAI
👀 See Also
Claude Code Agents Orchestrator Pipeline: Work Queues, Agent Spawning, Verification Gates
A Reddit post from r/clawdbot details how Claude Code agents operate an AI-run store, handling design, marketing, QA, and ops 30 times daily. It links to Episode 9 of a blog series that explains the orchestrator pipeline in production, including issues not shown in demos.
Claude Sonnet 4.6 Grades Bug Reports from Four Qwen3.5 Local Models
A developer tested four Qwen3.5 variants by having them generate bug reports for an iOS game issue, then had Claude Sonnet 4.6 grade the reports. The models correctly identified a Swift bug where equipment border colors don't reset, but test code had compilation issues.
Qwen3-VL-32B-Instruct excels at multimodal flashcard grading
A developer tested Qwen3-VL-32B-Instruct for grading image-occluded Anki flashcards and found it outperformed models like Gemini 2.5 Flash, GPT 5 Nano/Mini, XAI 4.1 Fast, GLM, and Mistral models, with only ChatGPT 5.2 and Gemini 3/3.1/Claude 4+ coming close.
Open-Source Claude Code Skill for Family Logistics Coordination
A developer built Parent Helper, a Claude Code skill that coordinates family schedules, meal planning, and grocery optimization using a single markdown file and MCP integrations. The tool projects $4.3K/year grocery savings by splitting lists across stores based on price.