Should I snatch up 16GB more RAM?
So my specs are:
CPU: i5 12th Gen
GPU: 3070
RAM: 16GB
I’ve considered a number of options for upgrading, and adding some more RAM seems to be the simplest & cheapest option for the potential improvements; 8GB RAM chips seem to still be reasonably priced, I have 2 free RAM slots, performance would experience an objective increase, and it would perhaps allow me to start effectively using self-hosted AI systems instead of dropping $20/month on Claude. But…is this advisable, and is it really as simple as plugging the extra RAM in? It’s a homebuilt, but it’s been a good few years since I really fiddled around with the internals.
9 replies
Buying RAM in this economy? Did you get a recent large inheritance?
I have 64gb ddr5 in my gaming box I built right before LLM-flation, and I'm considering selling 32gb for a massive profit
I had an entry level GPU (Radeon RX580 4GB) which I bought from a friend for €150 circa 2018, when its price was about twice more (won’t recall, from €200 to €250 maybe), he just did not need it and the card was overkill for my ‘just render the interface and occasional Photoshop-like kind of task’ workload. I thought I needed it, but it turned out I never utilised it. I don’t play games, that was for my Hackintosh to get a native support (which worked). I needed to upgrade the PSU alongside the GPU. I saw almost no difference in day to day life, since most of my work was some office-like stuff.
We entered some crazy times with GPUs a bit later, 2019 or ’20, another friend who was buying and selling these cards, told me I should sell it for €150 to €180 (basically returning my money back), and while I agreed with him at the time, I did not sell it. I didn’t want to spend my efforts on that, and then spend my efforts on buying something else (I thought I’d still need some GPU).
I regret that slightly. Not much, since €150 isn’t much to lose, and I did not have much resources to spend on this task. But if it was today, I think I’d do that.
I believe RAM prices would fall significantly at some point (would take years), I did buy +16 GB of DDR3 RAM right before the Rampocalypse, but I don’t see DDR3 being too expensive today. I don’t really utilise 32 GB vs 16 GB, but there’s not much point to sell it for what I’ve bought (€30 or so).
DDR5, on the other hand, may sell for a very good price, so if you don’t utilise it fully, why not then? You may buy a much better memory in some years. Keep in mind though that this crisis may take longer than we all hope. May take it not a year or two, but, say, five or ten.
Me, I’m ready for this. I run Linux, which works ideally with even just 8 GB of DDR3 RAM (my current laptop), so I’m pretty confident I’d survive till the prices drop. I’m not even sure I need it being newer and faster. Only if the hardware upgrade price would be insignificant to notice. I’ve got my current laptop from work for free, so that’s even better than I expected. But that’s just a different story.
Just make sure they're the same speed and size as your existing sticks and it should be fine.
Upgrading your system is generally as simple as plugging in 2 more compatible sticks, however, it is very unlikely to be the difference between whether or not you can run a local model
Depends. You didn't share if it's DDR4 or DDR5. I believe 11-14th gen supported both, depending on your motherboard, so the cost will vary. DDR4 is still cheaper, so much more worth buying more.
Edit: oh yeah, and you need to buy additional RAM that matches your current speed, otherwise you can degrade performance. Like 3200MHz or 5600MHz, etc.
Well do you regularly hit the RAM limit?
Since you obviously can’t use unified RAM, VRAM is going to make the most notable difference in locally run LLMs. But even if you do get a more powerful GPU, unless you drop thousands of dollars on one with more VRAM (on top of other components you may need to upgrade as well), you are, unfortunately, not going to have anything comparable to frontier cloud models.
The $20 a month is going to be far more affordable. Again, this is all by design so that you are locked into cloud models once you get hooked onto LLMs.
I guess the options would be to either increase VRAM by replacing the 3070 (8GB VRAM), or adding system RAM. In all fairness, system RAM for local AI can be really slow, so token/s output benefits much more from VRAM. Then again, I guess upgrading to a 16GB VRAM card is more expensive. Any chance to snatch up a cheap used 16GB video card?
If you upgrade system RAM, be sure to get a set with similar capacity and speed as you currently have in your system to avoid problems.