This new Chinese AI Is outperforming ChatGPT — and its open-source & runs locally
https://www.tomsguide.com/ai/this-new-chinese-ai-is-outperforming-chatgpt-and-it-runs-locallyOpen linkView original on futurology.todayhttps://www.tomsguide.com/ai/this-new-chinese-ai-is-outperforming-chatgpt-and-it-runs-locallyOpen linkView original on futurology.today
Ah I'll just pop it in the ol' Raspberry Pi then, easy peasy.
Lol, "runs locally". I mean, Claude rubs locally too if you're in the room with the racks.
Edit: I said what I said. Get some lube and go hang out with Claude's hot, noisy, 5kw rack. You know what they say "Once you go stack you never go back."
RPi 6: 256GB RAM
Currently vibe coding with it running on my vape!
Basically they never had any moat to begin with but no one else seems to know how to fit that much intelligence into less space. It's possible that it just fundamentally has to take up that much space which would also imply that future Computing gainss are going to be more focused on memory than raw competition
saved you a click
The really crazy thing is that this model still performs well at one-bit quantization, which shows it’s got a lot of room for improvement on size. It’s within an order of magnitude of being able to be run on consumer hardware, which would be an even more amazing kick in the balls to American AI companies.
Sucks that people lump AI into a single category of whatever cloud-hosted subscription that tech bros from Silicon Valley are pushing.
Given how memory is the bottleneck especially at the very low end it makes me wonder if one bit quantization of an extremely large model would be a gigabyte per gigabyte of ram better
C'mon, pop!