Ai2/AllenAI released a FOSS LLM which runs locally on iPhone
https://allenai.org/blog/olmoe-appOpen linkView original on lemmy.mlSyndicated from the fediverse. Read and engage on the original instance.
View original on lemmy.mlhttps://allenai.org/blog/olmoe-appOpen linkView original on lemmy.ml
12 replies
GitHub here: https://github.com/allenai/OLMoE.swift
Ai2 make as much of their training data available as possible: https://huggingface.co/datasets/allenai/OLMoE-mix-0924
I wonder how bit it is, normally they call them something with 3B or 7b
It’s 7 billion parameters big.
Ah, nice, not bad for a iPhone model.
Wanna see how you git clone and compile it on ios
Why only iPhone?
It probably uses Apple’s CoreML. Not sure if Android has an equivalent API.
https://developers.google.com/ml-kit/
Explain it pls
It's Apple's API for hardware-accelerated machine learning. Their new devices have specialized ML hardware so it's actually feasible to run on mobile devices. Probably what they also use for Apple Intelligence. https://developer.apple.com/documentation/coreml
Seems like a good project to implement on the green side (Or pinguin side)
according to the linked article, it sounds like it is for performance reasons.