Local LLM Inference Optimization: The Complete Guide
Good overview IMHO.
https://carteakey.dev/blog/local-inference/local-llm-optimization/Open linkView original on sh.itjust.worksSyndicated from the fediverse. Read and engage on the original instance.
View original on sh.itjust.workshttps://carteakey.dev/blog/local-inference/local-llm-optimization/Open linkView original on sh.itjust.works
2 replies
Huge red flag. XMP is not designed for use with memory-intense workloads like running LLMs.
I have a PC that has been crashing a lot recently, but only when running long RAM+CPU and RAM+GPU intensive tasks. It uses about 60%+ of ny RAM.
I ran a memory test for 9 passes, no errors. No errors with the CPU only, and no errors with GPU only.
So it can be the XMP that crashes it?