Favorite local llm budget rigs?
Mine is a Dell Precision T7910 tower with a Tesla P40 running Gemma 4 26b a4b it at 64k context.
The card is on the way, but by all measurements it should work great. It was about $900.
Syndicated from the fediverse. Read and engage on the original instance.
View original on lemmy.todayView original on lemmy.today
4 replies
What model do u plan to run?
Gemma 4! I’ve been testing it out and it has great results across a variety of applications.
Give onrith 35b a3b a try. On my test suite it makes Gemma 4 look bad. https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B
I run on an rx7600 with 8gb vram, 64gb ddr4 and a 5700x
Thank you I will!