42x Faster Prompt Lookup Drafting in llama.cpp
https://jadidbourbaki.github.io/blog/prompt-lookup-llama-cpp/Open linkView original on discuss.tchncs.deSyndicated from the fediverse. Read and engage on the original instance.
View original on discuss.tchncs.dehttps://jadidbourbaki.github.io/blog/prompt-lookup-llama-cpp/Open linkView original on discuss.tchncs.de
1 reply
It's interesting that neural graphs have locality at all. At least I find it surprising.