Spyke

Replies

Comment on

Ok, time to move from Ollama + OpenWebUI

OpenWebUI works with plain llama.cpp

16 is a bit small so try a MoE (e.g. QWEN 3.6 35BA3B) model and put experts on the CPU (although DDR4 may be underwhelming) which you can do with llama ( with offloading and drafting for T/s) but not ollama (spitting noise). Here's a good starting point. You'll likely get 60+T/s on say a 6 bit quant.

You can use a container approach, but llama.cpp is a bit of a moving target, with new cool features coming along regularly to support new models. I build it in a distrobox and running it is a simple call. When it doesn't want to build anymore because dependencies have changed too much, I just spin up a new distrobox and leave the old one there for older models. I find it a good balance between flexibility and ease of maintenance, and technically it's also a container approach. Take notes so you know how to set up the new one.

privacy

Comment on

It’s just become impossible to de-Google from Volkswagen, say GrapheneOS users

Impossible, no, the app is optional, car still drives, just not sucking data for VW (or not if you disable its network in gOS). It is however grounds for a lawsuit seeing as they're arbitrarily removing functionality that was likely advertised. Yoti is UK specific I believe, and those companies are little loss anyway, more of a prompt to divest. It's an unhealthy pattern, sure, but the sky is not falling.

The cultural effect of Android requiring developer registration and otherwise trying to kill third party stores like F-Droid, to get their precious walled garden, is a much more significant threat.

Comment on

Damn right

The customer is always right is what management says when they want you to be subservient to the customer because they believe it gets sales, or maybe just for the bosses amusement. Sacrifice dignity, agency, relevant subject knowledge. Given the choice between 'yes, massa' and an actual informed discussion on the merits of options, the latter every time.