Comment on
Poor little Qwen
Try Ternary Bonsai. It's a heavily quantized version of Qwen 3.8 27b. It won't do wonders, but it shoud be better: https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf-dev The page says you need a fork of llama.cpp, but the support was included in the latest release (0.7). You can read more about the mode on the main model page: https://huggingface.co/prism-ml/Ternary-Bonsai-27B-gguf