Spyke

Syndicated from the fediverse. Read and engage on the original instance.

AI and You Against the Machine: Guide so you can own Big AI and Run Local - YouTube

I've been following the work that went into this video for a couple of months and have grown to love Level1Techs.

Check out their forum and especially Ubergarm

5 replies

Ok, what machine is it? DL;dw

Their setup is here:

Oh, so an NVIDIA card with 24GB VRAM

Yep, and 2x64gb RAM

This is an excellent guide, way above most YT guides. ik_llama.cpp is a small miracle, and its a crime so few know of it.