GGUF Model Merging Script and Workflow for Qwen3.5-35B Variants

A Reddit user has shared a Python script and workflow for merging GGUF model files with minimal loss, specifically targeting Qwen3.5-35B variants. The approach combines two existing models: HauhauCS's Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive and samuelcardillo's Qwen3.5-35B-A3B-Claude-4.6-Opus-Reasoning-Distilled-GGUF.

Technical Details

The merged model is available as a Q4_0 quantized version at Hugging Face. According to the source, samuelcardillo's finetune outperforms Jackrong's version for Qwen 3.5 35B.

Merging Workflow

The Python script (available on Pastebin) was "vibecoded via Claude Opus 4.6" and supports:

• Merging GGUF files on Google Colab Free Tier
• Quantization via llama-quantize
• Q4_K_M quantization for 35B models
• Q8 quantization for 8B models

The author notes they can't create Q8_0 or F16 quantized versions due to disk space limitations on Google Colab Free tier, but suggests others can tweak the script via Claude Opus for those quantizations.

Optimal Settings

For best performance in LM Studio, use these parameters:

Temperature: 0.7
Top K Sampling: 20
Presence Penalty: 1.5
Top P Sampling: 0.8
Min P Sampling: 0
Seed: 3407 or 42

The system prompt (full version on Pastebin) should include this first line: "You are Qwen, created by Alibaba Cloud. You are a helpful assistant." The author notes the model underperforms without this line.