model : Ling-3.0-tiny (8b)
a lightweight hybrid reasoning MoE model with 7.9B total parameters and only 1.3B activated parameters per token. It is designed to deliver strong reasoning and agentic capabilities under a small inference compute footprint, making advanced model capabilities more accessible for local and resource-constrained deployment.
https://huggingface.co/inclusionAI/Ling-3.0-tinyOpen linkView original on lemmy.ml
8 replies
anyone try this? this might be good for my crappy laptop lol
is it good enough to use with Zoo Code? is it better than Qwen 3.5 4b?
EDIT: woa
https://artificialanalysis.ai/models/ling-3-0-tiny
But not yet supported in llama.cpp https://github.com/ggml-org/llama.cpp/pull/26608
Would this run on a cpu only pc? Would it be crazy slow if so?
Looking for some small model to cut my teeth on and I only have my laptop at the moment.
I've run Qwen 3.5 4b and Gemma 4 e2b on CPU only, this should be faster than those I think (fewer active parameters). If you have AVX512 or AVX10 then it should help a bit. Still slow compared to a GPU lol.
How slow 😁😅?
my laptop is crappy, so like 5 tokens per second lol, prompt processing of like 20 tokens per second
I think a decent laptop nowadays, even running CPU only, could probably do like 5x faster
Thanks for the info!
I haven't tried it yet, but I tested one of their previous ones with CPU interference on an Intel n97, and it was one of the best in terms of t/s performance and also prompt response quality on the metrics I tested against. When this run on llama.cpp I will try and give it a test.
personally,
i did one test of ling3-flash and gemma-4-31B side-by-side .
ling3 understood me and had a fantastic answer. gemma must have misunderstood what i was saying.. it wrote a long story-like paragraph that didnt answer my question. but it kinda had 1 bit of insight.
so: (maybe..) don't sleep on ling models !