Rules

1. Please tag [not libre software] and [never on-device] services as such (those not green in the License column here).
2. Be useful to others

Resources

github.com/ollama/ollama
github.com/danny-avila/LibreChat
github.com/Aider-AI/aider
wikipedia.org/wiki/List_of_large_language_models

Anyone tried these two for coding or agentic tasks? How do they compare? I know 27b is a dense one and very smart, but what is your experience and opinion?

ggml.ai joins HuggingFace

ggml.ai is better known as the entity behind llama.cpp. It's nice to hear good news! Thank you @ggerganov and @huggingface

https://github.com/ggml-org/llama.cpp/discussions/19759

@llm

Orange Pi closes the year by unveiling new details about the Orange Pi AI Station, a compact board-level edge computing platform built around the Ascend 310 series processor. The system targets high-density inference workloads with large memory options, NVMe storage support, and extensive I/O in a small footprint.

The AI Station is powered by an Ascend 310 series processor integrating 16 CPU cores clocked at up to 1.9 GHz, along with 10 AI cores running at up to 1.08 GHz and 8 vector cores operating at up to 1 GHz.

https://chat.qwen.ai/

https://wan.video/

https://wan.video/

Hello,

I have been looking into a new laptop, and was coming across ones with these NPUs heavily advertised in them. Doing some reading, they don't seem extremely functional at this stage.

They are around 45-50 TOPS at the highest it seems. I found some articles and comments suggesting that 'could' be useful for locally using smaller models, but also statements conflicting with that. As well, most, if not all, 'technical use' of them seem locked into the Windows environment. Even some program from AMD allowing local LLM use requires a Windows Server for it to communicate with, iirc. (AMD, GAIA)

So, is there, currently, any technical use for these, such that it makes much sense to grab a device with one for tinkering?

I'd considered experimenting with smaller models and seeing what comes of those (if small model improvements come through as DeepSeek proponents might suggest).

I'm also just generally new to the technology, but intrigued by the potential to localize usage; not only because of the potential to limit the environmental impact of large data center use.

Any comments, ideas, suggestions, or general pointing in a direction is very appreciated.

Thank you for taking the time. Have a good day!

Hallucinations are destroying under-resourced languages

These have been abundant even before #GenAI, when they were generated by machine translation. And for whatever motivation naive users have flooded crowdsourced resources with such hallucinations.

https://www.technologyreview.com/2025/09/25/1124005/ai-wikipedia-vulnerable-languages-doom-spiral/

@llm

Yesterday I had a brilliant idea: why not parse the wiki of my favorite table top roleplaying game into yaml via an llm? I had tried the same with beautfifulsoup a couple of years ago, but the page is very inconsistent which makes it quite difficult to parse using traditional methods.

• https://dsa.ulisses-regelwiki.de/Kul_Auelfen.html
• https://dsa.ulisses-regelwiki.de/erw_zauber_sf.html?erw_zaubersf=Alchimieanalytiker
• https://dsa.ulisses-regelwiki.de/KSF_Alter_Adersin.html

However, my attempts where not very successful to parse with a local mistral model (the one you get with ollama pull mistral) as it first insisted on writing more than just the yaml code and later had troubles with more complex pages like https://dsa.ulisses-regelwiki.de/zauber.html?zauber=Abvenenum So I thought I had to give it some examples in the system prompts, but while one example helped a little, when I included more, it sometimes started to just return an example from the ones I gave to it via system prompt.

To give some idea: the bold stuff should be keys in the yaml structure, the part that follows the value. Sometimes values need to be parsed a bit more like separating pages from book names - I would give examples for all that.

Any idea what model to use for that or how to improve results?

Since openai removed access to 4.5, I am looking for something comparable from any other company.

Personally, I used it when 4o was not good enough. 4.5 was way better at research and doing more complex programming tasks.

What is comparably good in your experience?

cross-posted from: https://lemmy.world/post/32632017

Is there a good latex ocr model that I can run with Ollama?

I have tried https://github.com/lukas-blecher/LaTeX-OCR but the result is just not promising. Is there a good Open Source latex ocr tools that I could use with ease?

I am hoping that there is a ollama llm with image support that specializes on latex ocr.

thanks a lot in advance!

They cry when companies profit from their work, while ignoring the most blatant solution from the start: the AGPL.

Now, its libre software license text file has been replaced with a fake, banning us users from freely forking new versions.

Open WebUI v0.6.6+ ... now adds a ... branding ... clause.

The original BSD-3 license continues to apply for all contributions made to the codebase up to and including release v0.6.5.

Open WebUI lets you download and run large language models (LLMs) on your device using Ollama.

Install Ollama

See this guide: https://lemmy.world/post/27013201

Install Docker (recommended Open WebUI installation method)

1. Open Console, type the following command and press return. This may ask for your password but not show you typing it.

sudo pacman -S docker

1. Enable the Docker service [on-device and runs in the background] to start with your device and start it now.

sudo systemctl enable --now docker

1. Allow your current user to use Docker.

sudo usermod -aG docker $(whoami)

1. Log out and log in again, for the previous command to take effect.

Install Open WebUI on Docker

1. Check whether your device has an NVIDIA GPU.
2. Use only one of the following commands.

Your device has an NVIDIA GPU:

docker run -d -p 3000:8080 --gpus all --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:cuda

Your device has no NVIDIA GPU:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Configure Ollama access

1. Edit the Ollama service file. This uses the text editor set in the $SYSTEMD_EDITOR environment variable.

sudo systemctl edit ollama.service

1. Add the following, save and exit.

[Service]
Environment="OLLAMA_HOST=0.0.0.0"

1. Restart the Ollama service.

sudo systemctl restart ollama

Get automatic updates for Open WebUI (not models, Ollama or Docker)

1. Create a new service file to get updates using Watchtower once everytime Docker starts.

sudoedit /etc/systemd/system/watchtower-open-webui.service

1. Add the following, save and exit.

[Unit]
Description=Watchtower Open WebUI
After=docker.service
Requires=docker.service

[Service]
Type=oneshot
ExecStart=/usr/bin/docker run --rm --volume /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once open-webui
RemainAfterExit=true

[Install]
WantedBy=multi-user.target

1. Enable this new service to start with your device and start it now.

sudo systemctl enable --now watchtower-open-webui

1. (Optional) Get updates at regular intervals after Docker has started.

docker run --rm --volume /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once open-webui

Use Open WebUI

1. Open localhost:3000 in a web browser.
2. Create an on-device Open WebUI account as shown.

I'm running ollama with llama3.2:1b smollm, all-minilm, moondream, and more. I am able to integrate it with coder/code-server, vscode, vscodium, page assist, cli, and also created a discord ai user.

I'm an infrastructure and automation guy, not a developer so much. Although my field is technically devops.

Now, I hear that some llms have "tools." How do I use them? How do I find a list of tools for a model?

I don't think I can simply prompt "Hi llama3.2, list your tools." Is this part of prompt engineering?

What, do you take a model and retrain it or something?

Anybody able to point me in the right direction?