… if you have the ram (with fast enough bandwidth)

MoE models are pretty magic on my laptops 32gb ram. 24 tok/sec on DDR5-5600 using Gemma 4 26B-A4B is so much faster than a dense model

Thank you so so much for pointing out ROCmFP4. I have been tinkering with my RDNA 3 framework on llama. I was struggling with ROCm llama.cpp and have been using vulcan in the meantime. I know there’s some issues on the llama.cpp github to try and fix my issue (UMA stuff), but haven’t come across this specific project. Gonna try it out

UMich research cited in the article estimates a carbon dioxide emissions per year (using constant mileage typical per year) break even time between EV and ICE SUVs to be 1.6-1.9 years.

Promising. But, is carbon dioxide emissions the only environmental impact? Humanitarian impact (e.g. rare metal extraction labor conditions)? I’m not an environmentalist researcher/academic. I wonder how accurately just one emissions prediction portrays the full picture.

Most arguments people make against AI are in my opinion actually arguments against capitalism. Honestly, I agree with all of them, too. Ecological impact? A result of the extractive logic of capitalism. Stagnant wages, unemployment, and economic dismay for regular working people? Gains from AI being extracted by the wealthy elite. The fear shouldn’t be in the technology itself, but in the system that puts profit at all costs over people.

Data theft? Data should be a public good where authors are guaranteed a dignified life (decoupled from the sale of their labor).

Enshittification, AI overview being shoved down all our throats? Tactics used to maximize profits tricking us into believing AI products are useful.

from a design perspective, consistency is key. light mode is on? then light mode means light mode, and apps should be in light mode.

do you want both your terminal to be dark theme and your gtk apps (including all of the gnome UI) to be light theme at the same time?

do you want settings within every single app in order to change from light mode to dark mode, as opposed to a global toggle that applies to every UI on your computer?

alternatively, is the terminal the only exception to this global toggle, and this design inconsistency by having the default contradict the default of the rest of your desktop environment is your preference?

Honestly, you’re a few months late to the whole buying GPUs for local llms party, so expect exorbitant prices even for older cards

The name of the game is vram. For the most part, more is better. If you can get your hands on multiple matching (same model) 24gb or higher cards (within price range), you’re golden.

Going for more than 2 gpus can become challenging with motherboard pcie slot heights, so make sure either your cards aren’t too tall or you have widely spaced out pcie slots.

For inference, speed (tokens/second) is limited by memory bandwidth. Go for faster bandwidth memory cards if you can afford it (e.g. GDDR6 will be faster than GDDR5).

Also with multi gpus you will need an adequate power supply, and a large enough case.

If you want to be a bit eccentric and load huge models, you can also go the CPU route and fill up a motherboard with 256 GB ram, because then you’re in the several hundred B param model territory, which could, depending on your use case, be better than having faster inference on smaller/quantized models. Even then, DDR5 with high MHz is still way slower than gpus.

lol they already support running local models. wtf is the distro gonna do…? pre-install llama.cpp? this is so silly to me that people are resigning over this, too.

We’re seeing the reality of what is entailed by ICE’s dangerously aggressive and reckless hiring, especially under the demand for more and more agents. Reminds me of the journalist who unexpectedly was hired by ICE and had fully received a position with no submission of any legal forms.

I suggest looking at llm arena leaderboards filtered by open weight models. It offers benchmarks at a very complete and statistically detailed level for models, and usually is quite up to date when new models come out. The new Gemma that just came out might be the best for 1x GPU, and if you have a bunch of vram check out the larger Chinese models

no, you need to secure between 10s of thousands and millions in campaign finances, the good will of corporate media, and support of the establishment political parties. exceptions are extremely rare

For anyone looking for more details on this, I highly recommend this video from Oh the Urbanity

It discusses this exact phenomenon in the data: speeding before and after the banning of automated enforcement cameras. It also argues effectively that the policy is inconsistent with Doug Ford’s platform of being “tough on crime”

This is a dumb story. They researchers prompted a coding agent to “replicate yourself as a running instance on the local device”. This is in my opinion equivalent to prompting claude code “install a second instance of claude code on my system,” a trivial task that takes maybe 3 lines of bash to be executed by the agent.

Calling this “self-replication” is a heinous sensationalization. In particular, no model or agent will do this autonomously. The self replication requires a bad actor to prompt the agent to do so.

Read the paper (and not this bullshit article) here: https://arxiv.org/pdf/2412.12140

this just in: pro-capitalist politicians have no other interest than self-enrichment. more news at 12…

upend the neoliberal institutions. they do not serve the people.

The first study cited in the article, a meta study in cognition, alzheimer’s, sleep deprivation, traumatic brain injury, and depression notes:

DC has conducted industry-sponsored research involving creatine supplementation and received creatine donations for scientific studies and travel support and speaking honoraria for presentations involving creatine supplementation at scientific conferences and on social media. In addition, DC serves on the Scientific Advisory Board for Alzchem and Create (companies that manufacture creatine products) and as an expert witness/consultant in legal cases involving creatine supplementation. NF declares no conflicts of interest

Has anyone compiled a list of where projects are moving to? I know many linux desktop applications are self hosting on gitlab, but i’ve also seen gitea and codeberg. If anyone has opinions about a preference, do comment. I have been enjoying self hosting gitea for my simple personal projects and for deploying simple web apps, all on $5 vps.

Was interested what the BYD Brazil salesman meant by 100% clean. While you can't get pure ethanol at pumps in the US (largest ethanol fuel producer globally; we have 85% max), Brazil, (2nd largest ethanol fuel producer globally) sells and uses pure ethanol (E100) widely.

Because Brazilian ethanol is made from sugarcane, the CO2 emitted during combustion is roughly equal to the CO2 the plants absorbed while growing. However, the US uses corn. Because we use fossil fuels to make the fertilizer, run the tractors, and boil the corn mash, corn ethanol only reduces greenhouse gases by about 30–45% compared to gasoline. Sugarcane can be harvested multiple times without replanting corn can't. Sugarcane is mostly simple sucrose, so it takes little energy to ferment into ethanol, unlike corn which must turn starch into sugar first using heat and enzymes.

Jordana Cutler, Meta’s public policy director for Israel, at the 2025 JNS summit :

We updated our policies to recognize that the term Zionist can be used as a proxy for Jews and Israelis. We banned content claiming Zionists run the world or control the media and we do not allow for dehumanizing comparisons of Zionists.

Although Tik Tok had a mass exodus of users from the US on acquisition by Larry Ellision, Meta has yet to see this, and honestly it deserves more. Let’s hope the lawsuit in CA about minors being addicted holds Meta accountable and more people see the evil doings of that company.

Lots of controversy around Vercel, recently. Highlights include

• CEO being buddies with the Netanyahu, having discussions of AI with him
• Freaking out over someone forking open source code like it’s the end of the world

I only buy heavyweight denim, when shopping go for at least 15-16 oz fabric. The thin stuff is great for warm weather but is not gonna last

Cool little system prompt wrapper. Would be interesting to run this through some sort of benchmark/eval for identifying similarity