Poor little Qwen
I felt sorry for it and stopped it, 14min and the poor thing was just going around in circles.
I'm just looking at running some local llm on my m2 16GB mac and this was with the default settings for the qwen3.5 9b in LM Studio. Any pointers to get the best out of a M2?
14 replies
If I recall properly, in system prompt you can set thinking level via keywords(quick google should do). And there are three levels "Xhigh, medium, low", Qwen defaults to Xhigh. So set it to something lower and maybe it will stop over thinking.
I like "I will assume 'Arcane' is a typo for 'docker'". I don't have any suggestions for you, but damn that poor LLM was so confused.
What is arcane? I also don't know
https://github.com/getarcaneapp/arcane
Just a Docker manager, I've moved from Portainer and slowly moving all my containers across.
Why? I don’t care for portainer’s commercial moves but it seems to work fine most of the time. How is arcane better?
Asking seriously. Not sarcastically.
Mostly because of their more and more restrictions on the free version.
Arcane seems to be the manager that is getting all the love at the moment, ie it was recommended a couple of times in the selfhosted community
Nice! Is it similar in features? A web app, an API, agents for managing multiple nodes, docker compose-in-the-browser support?
Try Ternary Bonsai. It's a heavily quantized version of Qwen 3.8 27b. It won't do wonders, but it shoud be better: https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf-dev The page says you need a fork of llama.cpp, but the support was included in the latest release (0.7). You can read more about the mode on the main model page: https://huggingface.co/prism-ml/Ternary-Bonsai-27B-gguf
Cries in old 1070 architecture
Oh god no. Bonsai is such a waste of time! Totally a dull version of Qwen. Keeps getting basic things wrong and isn’t agentic at all.
But I’d be curious to hear if you’ve used it and had a different experience.
I had already downloaded a Bonsai 27B, just not really tested it yet
What quant are you running? You might get better mileage from a gemma4 model of similar size and quants.
I had been using a gemma4 model, E4B and E2B. The E4B seems to work well
Yeah… E4B is definitely better than A2B.