Spyke

Replies

Comment on

Implementing a spellchecker on 64 kB of RAM back in the 1970s led to a compression algorithm that's technically unbeaten and part of it is still in use today

The real meat of the story is in the referenced blog post: https://blog.codingconfessions.com/p/how-unix-spell-ran-in-64kb-ram

TL;DR

If you're short on time, here's the key engineering story:

  • McIlroy's first innovation was a clever linguistics-based stemming algorithm that reduced the dictionary to just 25,000 words while improving accuracy.

  • For fast lookups, he initially used a Bloom filter—perhaps one of its first production uses. Interestingly, Dennis Ritchie provided the implementation. They tuned it to have such a low false positive rate that they could skip actual dictionary lookups.

  • When the dictionary grew to 30,000 words, the Bloom filter approach became impractical, leading to innovative hash compression techniques.

  • They computed that 27-bit hash codes would keep collision probability acceptably low, but needed compression.

  • McIlroy's solution was to store differences between sorted hash codes, after discovering these differences followed a geometric distribution.

  • Using Golomb's code, a compression scheme designed for geometric distributions, he achieved 13.60 bits per word—remarkably close to the theoretical minimum of 13.57 bits.

  • Finally, he partitioned the compressed data to speed up lookups, trading a small memory increase (final size ~14 bits per word) for significantly faster performance.

ukraine

Comment on

President Zelenskyy awarded the Order of Liberty to Vladyslav Heraskevych after the IOC disqualified him for wearing a helmet with photos of Ukrainian athletes killed by Russia.

I'm in awe how on point Ukraine's public messaging has been throughout this war. They've been perfectly measured and poignant about everything they've said.

Zelenskiy in particular has acted and spoken without a single misstep through all the challenges thrown his way. It's impressive...

Comment on

Natural selection at work

Context for this:

The common ancestor that links whales and deer existed millions of years ago and exhibited features shared by both groups. This ancestor likely possessed basic artiodactyl characteristics, such as an even number of toes on each foot and a certain bone structure within the ear. Over time, as these creatures diverged to inhabit distinct environments, their physical traits adapted accordingly. Despite these differences, the underlying genetic similarities persist, revealing their deep-rooted connection.

Comment on

Anon escapes from work

Reply in thread

Tragically, I think they only aim to be "do it somewhere else" nets. Because they introduce the threat of getting caught or partially caught and having to live with the embarrassment or wounds, respectively.

Comment on

Pokemon adventure

Reply in thread

"Retribution" is a common false friend (faux ami) in English-Spanish translation, where English speakers might mistakenly use retribución to mean punishment, whereas retribución actually means compensation, payment, or remuneration.

And Spanish is not the only language where "retribution" means remuneration. English is the weird one here.

memes

Comment on

I feel betrayed...

Reply in thread

Citing measurements made at the 1926 Iowa State Fair, they reported that the peak power over a few seconds has been measured to be as high as 14.88 hp (11.10 kW) and also observed that for sustained activity, a work rate of about 1 hp (0.75 kW) per horse is consistent with agricultural advice from both the 19th and 20th centuries [...]

Sounds to me like the 1 hp unit is fair, after all.

Comment on

‘Sputnik moment’: $1tn wiped off US stocks after Chinese firm unveils AI chatbot

Reply in thread

Text below, for those trying to avoid Twitter:

Most people probably don't realize how bad news China's Deepseek is for OpenAI.

They've come up with a model that matches and even exceeds OpenAI's latest model o1 on various benchmarks, and they're charging just 3% of the price.

It's essentially as if someone had released a mobile on par with the iPhone but was selling it for $30 instead of $1000. It's this dramatic.

What's more, they're releasing it open-source so you even have the option - which OpenAI doesn't offer - of not using their API at all and running the model for "free" yourself.

If you're an OpenAI customer today you're obviously going to start asking yourself some questions, like "wait, why exactly should I be paying 30X more?". This is pretty transformational stuff, it fundamentally challenges the economics of the market.

It also potentially enables plenty of AI applications that were just completely unaffordable before. Say for instance that you want to build a service that helps people summarize books (random example). In AI parlance the average book is roughly 120,000 tokens (since a "token" is about 3/4 of a word and the average book is roughly 90,000 words). At OpenAI's prices, processing a single book would cost almost $2 since they change $15 per 1 million token. Deepseek's API however would cost only $0.07, which means your service can process about 30 books for $2 vs just 1 book with OpenAI: suddenly your book summarizing service is economically viable.

Or say you want to build a service that analyzes codebases for security vulnerabilities. A typical enterprise codebase might be 1 million lines of code, or roughly 4 million tokens. That would cost $60 with OpenAI versus just $2.20 with DeepSeek. At OpenAI's prices, doing daily security scans would cost $21,900 per year per codebase; with DeepSeek it's $803.

So basically it looks like the game has changed. All thanks to a Chinese company that just demonstrated how U.S. tech restrictions can backfire spectacularly - by forcing them to build more efficient solutions that they're now sharing with the world at 3% of OpenAI's prices. As the saying goes, sometimes pressure creates diamonds.

Last edited 4:23 PM · Jan 21, 2025 · 932.3K Views