Llocalllama·LocalLLaMAbyLantier Qwen/QwQ-32B · Hugging Facehttps://huggingface.co/Qwen/QwQ-32BOpen linkView original on jlai.lu18Comments5
LLantier jlai.lu1Hide 1 replyGGUF quants are already out: https://huggingface.co/bartowski/Qwen_QwQ-32B-GGUF2
JJamonBear replysh.itjust.worksYay! let's try ollama run hf.co/bartowski/Qwen_QwQ-32B-GGUF:Q4_K_M /set parameter num_ctx 327681
suoko replyfeddit.it1Hide 1 replyWhy insane? For quality, speed, size? I find the coder 1.5b and 3b light and good1
mmorrowind replylemm.eeIt matches R1 in the given benchmarks. R1 has 671B params (36 activated) while this only has 323
GGUF quants are already out: https://huggingface.co/bartowski/Qwen_QwQ-32B-GGUF
Yay! let's try
ollama run hf.co/bartowski/Qwen_QwQ-32B-GGUF:Q4_K_M/set parameter num_ctx 32768insane, absolutely insane
Why insane? For quality, speed, size? I find the coder 1.5b and 3b light and good
It matches R1 in the given benchmarks. R1 has 671B params (36 activated) while this only has 32