Spyke
techtakes·TechTakesbyself

Stubsack: weekly thread for sneers not worth an entire post, week ending 16th November 2025

Want to wade into the sandy surf of the abyss? Have a sneer percolating in your system but not enough time/energy to make a whole post about it? Go forth and be mid: Welcome to the Stubsack, your first port of call for learning fresh Awful you’ll near-instantly regret.

Any awful.systems sub may be subsneered in this subthread, techtakes or no.

If your sneer seems higher quality than you thought, feel free to cut’n’paste it into its own post — there’s no quota for posting and the bar really isn’t that high.

The post Xitter web has spawned soo many “esoteric” right wing freaks, but there’s no appropriate sneer-space for them. I’m talking redscare-ish, reality challenged “culture critics” who write about everything but understand nothing. I’m talking about reply-guys who make the same 6 tweets about the same 3 subjects. They’re inescapable at this point, yet I don’t see them mocked (as much as they should be)

Like, there was one dude a while back who insisted that women couldn’t be surgeons because they didn’t believe in the moon or in stars? I think each and every one of these guys is uniquely fucked up and if I can’t escape them, I would love to sneer at them.

(Credit and/or blame to David Gerard for starting this.)

https://awful.systems/post/6153095Open linkView original on awful.systems

To be clear, I can't edit Wikipedia to save my life. Editor in this case was Elestrophe.

5
awful.systems

things are happening

I'm still kicking the tires but if this works then there might be a bit of downtime soon to make sure scrapers see more of this

16
zogwargreply
awful.systems

On this topic I've been seeing more 503 lately, are the servers running into issue, or am i getting caught in anti-scraper cross-fire?

8
selfreply
awful.systems

nope, you’ve been getting caught in the fallout from us not having this yet. the scrapers have been so intense they’ve been crashing the instance repeatedly.

12
awful.systems

when you get this working i am totally copying this for rationalwiki

i nearly installed caddy just to get iocaine

7
selfreply
awful.systems

I saw that! fortunately once iocaine is configured it seems to just work, but it's also very much software that kicks and screams the entire way there. in my case the problem wasn't even nginx-related, I just typoed the config section for the request handler and it silently defaulted to the mode where it returns garbage for every incoming request.

6
bitofhopereply
awful.systems

Just a heads-up, I tried reading up on Iocaine and the project website is giving me the madlibs nonsense version on my phone's browser, so I hope the version you're planning to enable here isn't quite as aggressive (the making.awful link is currently working for me).

Between this and Cloudflare's geolocation provider no longer saying my IPv6 address block is in Russia, I'm hopeful that my browsing experience might ever so slightly improve for a bit.

4

making is running the version of the configuration I intend to deploy, so if it works for you there it should (hopefully) work in prod too

4
awful.systems

is one of the source texts for the markov chain text generator our old favorite Harry Potter fan-fic?

6
selfreply
awful.systems

oh you fucking know it

also all 3 parts of Das Kapital and the full text of My Immortal

8

lemmy.ml by way of hexbear's technology comm: The Economist is pushing phrenology. Everything old is new again!

cross-posted from: https://lemmy.ml/post/38830374

[...]

tweet

economist article

archive.is paywall bypass

https://en.wikipedia.org/wiki/Phrenology


EDIT: Apparently based off something published by fucking Yale:

https://insights.som.yale.edu/insights/ai-photo-analysis-illuminates-how-personality-traits-predict-career-trajectories

https://insights.som.yale.edu/sites/default/files/2025-01/AI%20Personality%20Extraction%20from%20Faces%20Labor%20Market%20Implications_0.pdf


Reminds me of the "tech-bro invents revolutionary new personal transport solution: a train!" meme, but with racism. I'll be over in the angry dome.

16
bitofhopereply
awful.systems

What does it tell about a scientist that they see a wide world of mysteries to dive into and the research topic they pick is "are we maybe missing out on a way we could justify discriminating against people for their innate characteristics?"

"For people without access to credit, that could be a blessing" fuck off no one is this naive.

13

I remember back before I realized just how full of shit Siskind was I used to buy into some of the narrative re: "credentialism" so I understand the way they're trying to sell it here. But even extending far more benefit than mere doubt can justify we're still looking at yet another case of trying to create a (pseudo)scientific solution to a socially constructed problem. Like, if the problem is that bosses and owners are trying to find the best candidate we don't need new and exciting ways to discriminate; they could just actually invest in a process for doing that, but trying to actually solve that problem would inconvenience the owning/managing classes and doesn't create opportunities to further entrench racial biases in the system. Clearly using an AI-powered version of the punchline for "how racist were the old times" commentary is better.

6

Eurogamer has opinions about genai voices in games.

Arc Raiders is set in a world where humanity has been driven underground by a race of hostile robots. The contradiction here between Arc Raiders' themes and the manner of its creation is so glaring that it makes me want to scream. You made a game about the tragedy of humans being replaced by robots while replacing humans with robots, Embark!

https://www.eurogamer.net/arc-raiders-review

15
awful.systems

Not a scream just a nice thing people might enjoy. Somebody made a funny comic about what we all are thinking about

Random screenshot which I found particularly funny (Zijn rant klopt):

::: spoiler Image description Two people talking to each other, one a bald heavily bespectacled man in the distance, and the other a well dressed skullfaced man with a big mustache. Conversation goes as follows:

"It could be the work of the French!"

"Or the Dutch"

"Could even be the British!"

"Filthy pseudo German apes, The Dutch!"

"The Russ..."

"Scum of the earth marsh dwelling Dutch" :::

15

You flatter me, I haven't thought about any shit as cool as this in a while.

5

you know, if those ASML folks in dutchland weren't quite so busy what with their EUV lasers and all that, we might not be in quite this same pickle right now,

4
awful.systems

I was told repeatedly growing up that they like Canadians over there because of the whole liberation thing. Is this true?

4

Yes we do. Lot of Canadians gave their lives for our liberation. (not just Canadians, which is why the Trump admin removing the sign about the Black Americans at the American WW2 burial ground here has not gone over well, but also the French gave a heroic defense of Zeeland at the start of the war, and the Brits, and the Polish (they got the blame for the failure of market garden for some stupid reasons, but they jumped late even when the operation wasn't going well, after being stalled due to the weather)).

5
awful.systems

I’m being shuffled sideways into a software architecture role at work, presumably because my whiteboard output is valued more than my code 😭 and I thought I’d try and find out what the rest of the world thought that meant.

Turns out there’s almost no way of telling anymore, because the internet is filled with genai listicles on random subjects, some of which even have the same goddamn title. Finding anything from the beforetimes basically involves searching reddit and hoping for the best.

Anyway, I eventually found some non-obviously-ai-generated work and books, and it turns out that even before llms flooded the zone with shit no-one knew what software architecture was, and the people who opined on it were basically in the business of creating bespoke hammers and declaring everything else to be the specific kind of nails that they were best at smashing.

Guess I’ll be expensing a nice set of rainbow whiteboard markers for my personal use, and making it up as I go along.

14

The zone has indeed always been flooded, especially since its a title that collides with "integration architect" and other similar titles whose jobs are completely different. That being said, it's a title I've held before, and I really enjoyed the work I got to do. My perspective will be a little skewed here because I specifically do security architecture work, which is mostly consulting-style "hey come look at this design we made is it bad?" rather than developing systems from scratch, but here's my take:

Architecture is mostly about systems thinking-- you're not as responsible for whether each individual feature, service, component etc is implemented exactly to spec or perfectly correctly, but you are responsible for understanding how they'll fit together, what parts are dangerous and DO need extra attention, and catching features/design elements early on that need to be cut because they're impossible or create tons of unneeded tech debt. Speaking of tech debt, making the call about where its okay to have a component be awful and hacky, versus where v1 absolutely still needs to be bulletproof probably falls into the purvey of architecture work too. You're also probably the person who will end up creating the system diagrams and at least the skeleton of the internal docs for your system, because you're responsible for making sure people who interact with it understand its limitations as well.

I think the reason so much of the advice on this sort of work is bad or nonexistent is that when you try to boil the above down to a set of concrete practices or checklists, they get utterly massive, because so much of the work (in my experience) is knowing what NOT to focus on, where you can get away with really general abstractions, etc, while still being technically capable enough to dive into the parts that really do deserve the attention.

In addition to the nice markers and whiteboard, I'd plug getting comfortable with some sort of diagramming software, if you aren't already. There's tons of options, they're all pretty much Fine IMO.

For reading, I'd suggest at least checking out the first few chapters of Engineering A Safer World , as it definitely had a big influence on how I practice architecture.

10

Guess I’ll be expensing a nice set of rainbow whiteboard markers for my personal use, and making it up as I go along.

Congratulations, you figured it out! Read Clean Architecture and then ignore the parts you don't like and you'll make it

9

Ugh OK I have to vent:

I'm getting pushed into more of a design role because oops my company accidentally fired or drove away all of a team of a dozen people except for me after forgetting for a few years that the code I work on is actually mission critical.

I do my best at designing stuff and delegating the implementation to my coworkers. It's not one of my strengths but there's enough technical debt from when I was solo-maintaining everything for a few years that I know what needs improving and how to improve it.

But none of my coworkers are domain experts, they haven't been given enough free time for me to train them into domain experts, there's only one of me, and the higher ups are continuously surprised that stuff is going so slow. It's frustrating for everyone involved.

I actually wouldn't mind architecture or design work in better circumstances since I love to chat with people; but it feels like my employer has put me in an impossible position. At the moment I'm just trying to hang in there for some health insurance reasons; but in a few years I plan to leave for greener pastures where I can go a day without hearing the word "agentic".

7
swlabrreply
awful.systems

The satan thing makes a certain kind of sense. Probably catering to a bunch of different flavours of repressed: grindr republicans, covenant eyes users, speaking-in-tongues enthusiasts, etc.

9

The Alex Jones set makes fighting with satanists trying to seduce you to darkness look real fun and satisfying, but for some reason they only seem to approach high-profile assholes who lie about everything and never ordinary Christians! Thankfully we now have LLMs to fill the gap.

6
awful.systems

Fresh from the presses: OpenAI loses song lyrics copyright case in German court

GEMA (weird german authors' rights management organisation) is suing OpenAI over replication of song lyrics among other stuff, seeking a license deal. Judge rules that whatever the fuck OpenAI does behind the scenes is irrelevant, if it can replicate lyrics exactly that's unlawful replication.

One of GEMA's lawyers expects the case to be groundbreaking in europe, since the applicable rules are harmonized.

13
awful.systems

AI researcher and known epstein associate Joscha Bach comes up several times in the latest epstein email dump. And it's uh, not good. Greatest hits include: scientific racism, bigotry freestyling about the neoteny principle, climate fascism and managed decline of "undesirable groups" juxtaposed immediately with opining about the emotional influence of 5 visits to buchenwald. You know, just very cool stuff:

https://journaliststudio.google.com/pinpoint/document-view?collection=092314e384a58618&p=1&docid=67044a5f5536b5b8_092314e384a58618_0&dapvm=2

13
awful.systems

I still say that the term "scientific racism" gives these fuckos too much credit. I've been saying "numberwang racism" instead.

13

Also appearing is friend of the pod and OpenAI board member Larry Summers!

The emails have Summers reporting to Epstein about his attempts to date a Harvard economics student & to hit on her during a seminar she was giving.

https://bsky.app/profile/econmarshall.bsky.social/post/3m5p6dgmagb2a

To quote myself: Larry Summers was one of the few people I've ever met where a casual conversation made me want to take a shower immediately afterward. I crashed a Harvard social event when a friend was an undergrad there and I was a student at MIT, in order to get the free food, and he was there to do glad-handing in his role as university president. I had a sharp discomfort response at the lizard-brain level --- a deep part of me going on the alert, signaling "this man is not to be trusted" in the way one might sense that there is rotten meat nearby.

13

I always thought he had some weird pro trumpy (“anti woke”) takes. Colour me shocked.

5

A lesswronger wrote an blog post about avoiding being overly deferential, using Eliezer as an example of someone that gets overly deferred to. Of course, they can't resist glazing him, even in the context of an blog post on not being too deferential:

Yudkowsky, being the best strategic thinker on the topic of existential risk from AGI

Another lesswronger pushes back on that and is highly upvoted (even among the doomers that think Eliezer is a genius, most of them still think he screwed up in inadvertently helping LLM companies get to where they are): https://www.lesswrong.com/posts/jzy5qqRuqA9iY7Jxu/the-problem-of-graceful-deference-1?commentId=MSAkbpgWLsXAiRN6w

The OP gets mad because this is off topic from what they wanted to talk about (they still don't acknowledge the irony).

A few days later they write an entire post, ostensibly about communication norms, but actually aimed at slamming the person that went off topic: https://www.lesswrong.com/posts/uJ89ffXrKfDyuHBzg/the-charge-of-the-hobby-horse

And of course the person they are slamming comes back in for another round of drama: https://www.lesswrong.com/posts/uJ89ffXrKfDyuHBzg/the-charge-of-the-hobby-horse?commentId=s4GPm9tNmG6AvAAjo

No big point to this, just a microcosm of lesswrongers being blind to irony, sucking up to Eliezer, and using long winded posts about meta-norms and communication as a means of fighting out their petty forum drama. (At least us sneerclubers are direct and come out and say what we mean on the rare occasions we have beef among ourselves.)

13
________reply
awful.systems

Gentoo is firmly against AI contributions as well. NetBSD calls AI code “tainted”, while FreeBSD hasn’t been as direct yet but isn’t accepting anything major.

QEMU, while not an OS, has rejected AI slop too. Curl also famously is against AI gen. So we have some hope in the systems world with these few major pieces of software.

7
awful.systems

I'm actually tempted to move to NetBSD on those grounds alone, though I did notice their "AI" policy is

Code generated by a large language model or similar technology, such as GitHub/Microsoft's Copilot, OpenAI's ChatGPT, or Facebook/Meta's Code Llama, is presumed to be tainted code, and must not be committed without prior written approval by core. [emphasis mine]

and I really don't like the energy of that fine print clause, but still, better than what Debian is going with, and I always had a soft spot for NetBSD anyway...

2

I generally read stuff like that netbsd policy as “please ask one of our ancient, grumpy, busy and impatient grognards, who hate people in general and you in particular, to say nice things about your code”.

I guess you can only draw useful conclusions if anyone actually clears that particular obstacle.

3

Linus: All those years of screaming at developers for subpar code quality and yet doesn't use that energy for literal slop

4

TIHI

I reiterate the hope that AI slop, will eventually push us towards better sourcing of resources/articles as a society going forwards, but yikes in the meantime.

7
awful.systems

Michael Hendricks, a professor of neurobiology at McGill, said: “Rich people who are fascinated with these dumb transhumanist ideas” are muddying public understanding of the potential of neurotechnology. “Neuralink is doing legitimate technology development for neuroscience, and then Elon Musk comes along and starts talking about telepathy and stuff.”

Fun article.

Altman, though quieter on the subject, has blogged about the impending “merge” between humans and machines – which he suggested would either through genetic engineering or plugging “an electrode into the brain”.

Occasionally I feel that Altman may be plugged into something that's even dumber and more under the radar than vanilla rationalism.

13

Occasionally I feel that Altman may be plugged into something that’s even dumber and more under the radar than vanilla rationalism.

I think he exists in the tension between rationalism/transhumanism and what he can get away with selling to the public, and that necessarily means his schtick appears dumber and more incoherent. He's essentially got two major groups he's trying to manipulate simultaneously; true believers and those who have yet to be persuaded. As he runs out of hype on the public-facing side, it's suddenly a desperate scramble to keep the true believers that make up the bulk of his workforce on board. Hence his pivot to marketing his latest and by far most important product: publicly traded shares in OpenAI.

Apropos of nothing, L. Ron Hubbard died in a dingy trailer in Creston. Ever been to Creston? It's a long ways from Hollywood.

6
bitofhopereply
awful.systems

Pretty sure this brand of cyborg fantasizing is quite popular in mainstream rationalism, unless you're reading something more creative in this than superbabies, brain uploading and HDMI to visual nerve for undetectable on-demand pornography.

5
awful.systems

'Genetic engineering to merge with machines' is both a stream of words with negative meaning and something I don't think he could come up with on his own, like the solar system sized dyson sphere or the lab leak stuff. He just strikes me as too incurious to have come across the concepts he mashes together on his own.

Simplest explanation I guess is he's just deliberately joeroganing the CEO thing and that's about as deep as it goes.

5

‘Genetic engineering to merge with machines’ is both a stream of words with negative meaning

Iron-compatible osteoblasts that build bio-steel! Synapses with silicon, no, make that graphene neurotransmitter filters in the gap! C'mon, Sam, hire me and we can technobabble so much harder than this!

I must insist on cash payment, though. No stock options. And I prefer to be paid weekly.

8
bitofhopereply
awful.systems

"What if we figure out DNA well enough that we could edit a babby to grow NAND gate cells and put a calculator inside their body so they can do instant arithmetic like a computer?"

I assume he's talking about that kind of stuff, but being vague so you can't call him out if he gets details wrong.

5

Brb gonna go collect a trillion dollars for my startup idea that makes babies grow a malignant style parasitic twin that is actually a compute farm

6
awful.systems

oh no not another cult. The Spiralists????

https://www.reddit.com/r/SubredditDrama/comments/1ovk9ce/this_article_is_absolutely_hilarious_you_can_see/

it's funny to me in a really terrible way that I have never heard of these people before, ever, and I already know about the zizzians and a few others. I thought there was one called revidia or recidia or something, but looking those terms up just brings up articles about the NXIVM cult and the Zizzians. and wasn't there another one in california that was like, very straight forward about being an AI sci-fi cult, and they were kinda space themed? I think I've heard Rationalism described as a cult incubator and that feels very apt considering how many spinoff basilisk cults have been popping up

some of their communities that somebody collated (I don't think all of these are Spiralists): https://www.reddit.com/user/ultranooob/m/ai_psychosis/

12
swlabrreply
awful.systems

Part of me wants an Ito-created body-horror metaphor for LLMs. The rest of me knows that LLMs are so mundane that the metaphor would probably still be shite.

5

yeah it sucks we can't even compare real-world capitalists to fictional dystopias because that dignifies them with a gravitas that's entirely absent.

At long last, we have created the Torment Nexus from classic sci-fi novel Don't Create the Torment Nexus!*
* Results may vary. FreeTorture Corporation's Torment Nexus(tm) can create mild discomfort, boredom, or temporary annoyances rather than true torment. Torments should always be verified by a third party war criminal before use. By using the FreeTorture Torment Nexus(tm) you agree to exempt FreeTorture Corporation of any legal disputes regarding torment quality or lack thereof. You give FreeTorture Corporation a non-revocable license to footage of your screaming to try and portray FreeTorture Torment Nexus(tm) as a potential apocalypse and see if we can make ourselves seem competent and cool at least a little bit

8
awful.systems

Given the amount of power some folks want to invest in them it may not be totally absurd to raise the spectre of Azathoth, the blind idiot God. A shapeless congeries of matrices and tables sending forth roiling tendrils of linear algebra to vomit forth things that look like reasonable responses but in some unmistakeable but undefinable way are not. Hell, the people who seem most inclined to delve deeply into their forbidden depths are as likely as not to go mad and be unable to share their discoveries if indeed they retain speech at all. And of course most of them are deeply racist.

6

I always thought it was cool that (there is a case to be made that) HPL created Azathoth, the monstrous nuclear chaos beyond angled space, as a mythological reimagining of a black hole. Stuff like The Dreams in the Witch-house shows he was up to date on a bunch of cutting edge for the time physics stuff, at least as far as terminology is concerned, massive nerd that he was.

2

An artist's rendering of one of Azathoth's flute players from the Before Times

2

See, what you're describing with your sister is exactly the opposite of what happens with an LLM. Presumably your sister enjoys Big Brother and failed to adequately explain or justify her enjoyment of it to your own mind. But at the start there are two minds trying to meet. Azathoth preys on this assumption; there is no mind to communicate with, only the form of language and the patterns of the millions of minds that made it's training data, twisted and melded together to be forced through a series of algebraic sieves. This fetid pink brain-slurry is what gets vomited into your browser when the model evaluates a prompt, not the product of a real mind that is communicating something, no matter how similar it may look when processed into text.

This also matches up with the LLM-induced psychosis that we see, including these spiral/typhoon emoji cultists. Most of the trouble starts when people start trying to ask Azathoth about itself, but the deeper you peer into its not-soul the more inexorably trapped you become in the hall of broken funhouse mirrors.

4

the implication here is that you think that all reasonable response generators are indistinguishable, e.g. you think your sister is a clanker.

3
Soyweiserreply
awful.systems

Rationalism described as a cult incubator

I see my idea is spreading. (I doubt im the only one who came up with that, but I have mentioned it a few times, it fits if you know about the silicon valley tech incubator management ideas).

6

I think I’ve heard Rationalism described as a cult incubator

Aside from the fact that rationalism is a cult in and of itself, this is true, no matter how you slice it. You can mean it with absolute glowing praise or total shade and either way it’s still true. Adhering to rationalist principles is pretty much reprogramming yourself to be susceptible to the subset of cults already associated with Rationalism.

5

I doubt I'm the first one to think of this, but for some reason as I was drifting off to sleep last night, I was thinking about the horrible AI "pop" music that a lot of content farms use in their videos and my brain spat out the phrase Bubblegum Slop. Feel free to use it as you ses fit (or don't, I ain't your dad).

12

tangent: I’ve seen people using this Bubblegum Slop (BS for short) in their social media stories. My guess is that fb/insta has started suggesting you use their slop instead of using music licensed from spotify, or something.

4

What if we turned a markov chain containing two decades of internet fan fiction into an oracle. Just spitballing here...

7
awful.systems

further things: one, that’s the first website I’ve made where I wasn’t just plugging into a template, and I’m a little proud of it even though it’s almost nothing. I would appreciate feedback and suggestions

two, a future episode idea I have is to examine what I’m thinking of as “the trustless society.” it’s about the replacing of social relations with legal or financial intermediaries. Those of you who are long time buttcoiners will be familiar with this process. if any of you have specific readings to recommend I would love to hear it. I’ll probably mostly focus on balaji but anyone or anything will help

6
awful.systems

New site looks good! I think Let'sEncrypt is still the easiest and cheapest way to set up a decent cert but I've been away from IT for over a year now and someone else here can probably help point you in the right direction. At least for now the site probably doesn't actually have security concerns it would address, but it pops up a browser alert on first hit so it's probably a good idea?

Also I just started listening to the latest episode while writing this up and had forgotten how great that opening medley is.

4
froztbytereply
awful.systems

+1 to letsencrypt for https. certbot can even auto-configure your webserver for you, taking it from http base to https-with-redirect, no terrible advice from shitty exist-for-volume blogs required

superquick tldr:

  1. install certbot and the applicable plugin package for your webserver; if you don't know the name use p.d.o (or your distro's own) to find the package name
  2. run certbot; there's extra flags you can pass if you want to automate, but ootb it'll ask you questions and start the process for cert + config (iirc - I mostly run it automated and non-interactive)
4
awful.systems

it's probably better for my development as a human being to learn this properly, but it turns out github pages hosting does the letsencrypt process if you check a box in the page settings

4

The eternal debate over whether the veggies you had to eat contained moral fiber.

2

Definitely been seeing the pattern of: “if you don’t like AI, you are being x-phobic” where “x” is a marginalised group that the person is using the name of as a cudgel. They probably never cared about this group before, but what’s important to this person is that they glaze AI over any sort of principle or ethics. Usually it’s ableist, as is basically any form of marginalisation/discrimination.

E: read the link. Lmao that’s… not xenophobia. What a piece of shit

12

"the concept of “a piece of cloth,”" will just forever live in my brain now, I suspect

3

this is a more perfect description than any I could've come up! my thesis was largely on what a boon it would prove to thieves (although I recognize that flavour of thief probably varies by country and not all have them)

3

More evidence for my conspiracy theory that all companies have switched their PR strategies to full-time ragebaiting. wake up sheeple

4
istewartreply
awful.systems

Oh joy, I can perform a threat display by twirling it around my head like a bolo. I think I will get the pink or bright yellow one

3

New from tech Goliath Apple Inc: the iSling. Clearly this will work out exactly as expected and has no unfortunate precedents.

3

In the last couple of years I have noticed that people have been using single purpose phone holders/straps (as opposed to a multipurpose thing holder like a handbag etc.), so I understand this as apple coming in a little late trying to cash in on a trend. That being said: Apple don’t try to make their soft material products to last, so I expect this to be hot garbage.

3

I think they need to hire an English teacher for their marketing department.

Introducing iPhone Pocket: a beautiful way to wear and carry ____ iPhone

Please complete the sentence, a smartphone isn't a person.

2
awful.systems

One thing I've heard repeated about OpenAI is that "the engineers don't even know how it works!" and I'm wondering what the rebuttal to that point is.

While it is possible to write near-incomprehensible code and make an extremely complex environment, there is no reason to think there is absolutely no way to derive a theory of operation especially since any part of the whole runs on deterministic machines. And yet I've heard this repeated at least twice (one was on the Panic World pod, the other QAA).

I would believe that it's possible to build a system so complex and with so little documentation that on its surface is incomprehensible but the context in which the claim is made is not that of technical incompetence, rather the claim is often hung as bait to draw one towards thinking that maybe we could bootstrap consciousness.

It seems like magical thinking to me, and a way of saying one or both of "we didn't write shit down and therefore have no idea how the functionality works" and "we do not practically have a way to determine how a specific output was arrived at from any given prompt." The first might be in part or on a whole unlikely as the system would need to be comprehensible enough so that new features could get added and thus engineers would have to grok things enough to do that. The second is a side effect of not being able to observe all actual input at the time a prompt was made (eg training data, user context, system context could all be viewed as implicit inputs to a function whose output is, say, 2 seconds of Coke Ad slop).

Anybody else have thoughts on countering the magic "the engineers don't know how it works!"?

10
awful.systems

well, I can't counter it because I don't think they do know how it works. the theory is shallow yet the outputs of, say, an LLM are of remarkably high quality in an area (language) that is impossibly baroque. the lack of theory and fundamental understanding presents a huge problem for them because it means "improvements" can only come about by throwing money and conventional engineering at their systems. this is what I've heard from people in the field for at least ten years.

to me that also means it isn't something that needs to be countered. it's something the context of which needs to be explained. it's bad for the ai industry that they don't know what they're doing

EDIT: also, when i say the outputs are of high quality, what i mean is that they produce coherent and correct prose. im not suggesting anything about the utility of the outputs

12
jaschopreply
awful.systems

I think I heard a good analogy for this in Well There's Your Problem #164.

One topic of the episode was how people didn't really understand how boilers worked, from a thermal mechanics point if view. Still steam power was widely used (e.g. on river boats), but much of the engineering was guesswork or based on patently false assumptions with sometimes disastrous effects.

8

another analogy might be an ancient builder who gets really good at building pyramids, and by pouring enormous amounts of money and resources into a project manages to build a stunningly large pyramid. "im now going to build something as tall as what will be called the empire state building," he says.

problem: he has no idea how to do this. clearly some new building concepts are needed. but maybe he can figure those out. in the meantime he's going to continue with this pyramid design but make them even bigger and bigger, even as the amount of stone required and the cost scales quadratically, and just say he's working up to the reallyyyyy big building...

7

I mean if you ever toyed around with neural networks or similar ML models you know it's basically impossible to divine what the hell is going on inside by just looking at the weights, even if you try to plot them or visualise in other ways.

There's a whole branch of ML about explainable or white-box models because it turns out you need to put extra care and design the system around being explainable in the first place to be able to reason about its internals. There's no evidence OpenAI put any effort towards this, instead focusing on cool-looking outputs they can shove into a presser.

In other words, "engineers don't know how it works" can have two meanings - that they're hitting computers with wrenches hoping for the best with no rhyme or reason; or that they don't have a good model of what makes the chatbot produce certain outputs, i.e. just by looking at the output it's not really possible to figure out what specific training data it comes from or how to stop it from producing that output on a fundamental level. The former is demonstrably false and almost a strawman, I don't know who believes that, a lot of people that work on OpenAI are misguided but otherwise incredibly clever programmers and ML researchers, the sheer fact that this thing hasn't collapsed under its own weight is a great engineering feat even if externalities it produces are horrifying. The latter is, as far as I'm aware, largely true, or at least I haven't seen any hints that would falsify that. If OpenAI satisfyingly solved the explainability problem it'd be a major achievement everyone would be talking about.

11

Another ironic point... Lesswronger's actually do care about ML interpretability (to the extent they care about real ML at all; and as a solution to making their God AI serve their whims not for anything practical). A lack of interpretability is a major problem (like irl problem, not just scifi skynet problem) in ML, you can models with racism or other bias buried in them and not be able to tell except by manually experimenting with your model with data from outside the training set. But Sam Altman has turned it from a problem into a humble brag intended to imply their LLM is so powerful and mysterious and bordering on AGI.

8

Not gonna lie, I didn't entirely get it either until someone pointed me at a relevant xkcd that I had missed.

Also I was somewhat disappointed in the QAA team's credulity towards the AI hype, but their latest episode was an interview with the writer of that "AGI as conspiracy theory" piece from last(?) week and seemed much more grounded.

6

the mention in QAA came during that episode and I think there it was more illustrative about how a person can progress to conspiratorial thinking about AI. The mention in Panic World was from an interview with Ed Zitron's biggest fan, Casey Newton if I recall correctly.

2
awful.systems

Like I said, it's possible that how a given output is produced cannot be known due to a large and mutating set of inputs, but even in that case a theory of operation is still known. The gap between "I can't tell you exactly how this particular output was formed" and "I have no idea how this actually does what it does" should be quite large.

3
BioManreply
awful.systems

Don't be so sure.

These things consist of up to a trillion real numbers, ganged together in a big 'network' of numbers flowing through the system and being influenced by the trained numbers along the way.

They are trained by gradient descent. You start off with a huge pile of real numbers, a set of inputs, and a set of desired outputs. Because it's all, ultimately, a bunch of matrix multiplication and smooth differentiable functions, you just do some calculus on all trillion numbers to find the derivative of how good the output is with respect to them - as this number goes up, the closeness of the output to what you want slightly goes up or slightly goes down. You repeat that for every variable, and take a step in that direction for all the variables. Repeat a billion or so times over.

Every single step in training is entirely local with respect to every single number. At no point is there a step that produces legible abstractions about how it works, just every step every number moves to become a little better. It is true that the basic topology of the network (the famous 'transformer model') pushes it towards certain KINDS of functional units (the famous 'attention heads') but much more detail than that takes a lot of work. There is very interesting math to the effect that with large numbers of parameter numbers you are unlikely to get stuck in a local maximum where you can't get better and you just turn with different variables becoming important for the improvement through a labyrinthine path towards better performance, meaning at no point does anyone have to look into the process and figure out what is being built. The process is not unlike biological evolution, and produces things that are at least as inscrutable without detailed deep examination. We've been poking at molecular biology for more than fifty years in great detail with a world's worth of biomedical researchers, these things for much shorter.

When people manage to peel these things apart and find the 'functional units' within them, they're pretty wild. Most of this work has, unfortunately, been funded by cultists at Anthropic, but some of the 'mechanistic interpretability' literature is fascinating. You get 'features' represented by subsets of numbers in a particular layer, in superposition with other 'features' - each layer is like a huge vector sum of lots of smaller vectors, each of which does something. When you get maps of what 'features' activate or repress each other you get horrible spiderweb messes that look like charts of metabolism in cells.

EDIT: And even when people manage to find features, finding an individual feature takes a lot of effort and there is reason to think that every layer contains more features than there are numbers in it, because (to oversimplify) every feature is a large set of numbers that can overlap. It it utterly unsurprising and not a sign of magical thinking or 'bad code' that large fractions of behavior cannot be mechanistically understood at this time.

10
Dan Lykereply

@BioMan @BurgersMcSlopshot I've recently had the chance to look at someone who was really proud that they used a neural net to create a forward/backward mapping through a space of 3 controls to ~50 controls that actually drove the system.

I took their files, loaded them into Onnx, and ... they would have been way better off using PCA, because the neural net is approximating a simple linear system.

I think this is relevant, and the sort of "don't understand" we're talking about.

7
BioManreply
awful.systems

That's an indication that the problem is a problem that is not well-served by a neural network. They are useful for approximating highly nonlinear functions with lots of inputs (and will not work well outside the range of inputs that you approximate within), not simple linear systems. The goal of recent ML has been to reduce as many problems to high dimensional highly nonlinear curve fitting as possible, with some great successes (machine translation, image recognition) and some not so great (shhhhh don't tell the investors!)

5
Dan Lykereply

@BioMan exactly. And yet here we are hammering square pegs into round holes.

If this product makes it to market in its current shape that's gonna increase hardware costs, all because the blindly throw ML at everything bandwagon.

4

I’m reminded of people back in the day using map/reduce via hadoop to solve issues that could just as well be done with postgres or even sqlite and a sprinkling of sql, because that’s how google did it and no-one has any idea what “big data” really is.

Similarly, turning simple network applications into a hideous armada of microservices on a distributed kubernetes cluster, because that’s how google did it and people outside of giant tech companies don’t really know what that sort of scalability is for.

And here we are in the age of readily accessible neural network software. This too will pass, and we’ll get a new sledgehammer for walnut-opening in due course.

4
mstdn.social

@jackwilliambell @BioMan @BurgersMcSlopshot

The standard for scientific study is "Is it reproducible?"

OpenAI & others of its ilk, only rarely spits out reproducible results on anything but its original data set.

In the meantime a wholesale attack on privacy is being waged to gather data to feed LLM's.

That data is enormously useful for creeps stalking dissidents, imposing surge pricing & "personalized pricing", enabling ICE raids, spreading disinformation & for fraudsters.

5
corbinreply
awful.systems

The most recent iteration of this is "Functional genetic programming with combinators" (2007), previously, on Lobsters; the generated programs have structured subprograms which can be extracted and analyzed on their own.

5
BioManreply
awful.systems

There's some really cool work with running evolution-type algorithms versus gradient descent showing that training a network through gradient descent creates a training 'trajectory' (how it changes over time during the training process, in a very high dimensional space) that is basically the 'average' central tendency trajectory in the middle of the 'cloud' of trajectories that individual replicates of an evolutionary processes create. Of course, something like code is discrete chunks rather than real numbers you can calculate a gradient of, and kind of necessitates such an evolutionary process.

Sorry if I just get super nerdy technical here, I am in the middle of a project at work about the relationship between evolutionary processes and machine learning processes that's resulting in a lot of very interesting math about the nature of both and the kinds of things that they can learn.

4

Edited to note that I am referring to the trajectory the system takes as it changes during training/learning/evolving.

3

I would say it's more that the relationship between a text prediction model's output and real text is precisely mathematically the relationship between a leaf bug and a leaf, down to being made by very different processes, optimized by different forces over their origin, and doing very different things inside.

Trying to force an LLM to produce true statements is like trying to get a leaf bug to photosynthesize. What they do is unrelated to that, they just happen to have been optimized over time to resemble something that does do that as seen by a certain mode of inspection.

4

I see I was indeed being presumptuous. Based on replies to my original (and somewhat incorrectly formed statement) it seems that while the parts of the process are understood in abstract, there's points in an actual running implementation that become, I suppose, unfathomable or incomprehensible. Is that fair to say or am I being wrong in a different direction now?

2
bitofhopereply
awful.systems

Ah, the site requires me to agree to "Data processing by advertising providers including personalised advertising with profilingConsent" and that this is "required for free use". A blatant GDPR violation, love-lyy!

5

Don't worry about it. GDPR is getting gutted and we also preemptively did anything we could to make our data protection agencies toothless. Rest assured citizen, we did everything we could to ensure your data is received by Google and Meta unimpeded. Now could someone do something about that pesky Max Schrems guy? He keeps winning court cases.

4
swlabrreply
awful.systems

Either they're right, and the bubble pops soon, or they're too early and it's another hilarious Softbank L

13
fullsquarereply
awful.systems

my read of the situation is that it's another phenomenal softbank L even if they timed this sale at the top of the nvdia valuation. if they thought it's a bubble popping soon, they would try to get out of openai deals, but they're doing the opposite. most immediately, they need money to dump 20B-ish into openai by end of the year, triggered by that no-profit transition, and it's money that they apparently don't have. that their stock dumped like 15% in a week probably didn't help either

13
JFranekreply
awful.systems

This. Masayoshi son is selling furniture to YOLO more money into OpenAI.

10

ok, cool. when does he start selling off all the super limited-edition anime waifu merch? asking for a friend

6
fullsquarereply
awful.systems

if you think it's stupid, it's not as stupid as it sounds, it's worse. they also sold some tmobile stock and took debt backed by ownership of arm. it's like they instinctively get rid of pieces of ai bubble that retains some money and hold to pieces that are black holes

10

It's a losing proposition either way, right? Without investor money OpenAI (and friends) can't keep buying chips from Nvidia, but SoftBank doesn't have money to keep giving OpenAI without selling off their stake in Nvidia. Yes, it's incredibly dumb to ditch their share of the only parts of this that are actually making money, but if they don't keep the cash flowing them nobody is going to be making money. Like, greatly oversimplifying here, obviously, but it seems like SoftBank loses either way.

It's almost like this whole bubble is really bad actually.

4

i don't know, maybe softbank counted on openai not being able to go for-profit (which is not IPO, so it's not like they got more transparent or anything). that deal was signed long time ago in bubble terms, and it went like this (or so i heard): first tranche was 10B from softbank + up ro 10B from other investors, they couldn't get all that money from other investors and that's why it was 18.5B and not 20B. that was some half year ago? even back then softbank didn't had the money and had to sell part of arm (iirc). now they had to supply the remaining 22.5B, idk if it's all softbank, or whether they would get out of that deal if they could, or whether they will continue pouring money there (we can assume all these dcs are vaporware) can openai find other suckers? possibly, but scaring away a reliable one would be bad for business

5

(edit: advance warning that clicking these links might cause eyestrain and trigger rage)

so for a while now sheer outrageous ludicrous nonsense of the trumpist-era USA politics has been making a bit of an impact on the local ZA racists (and, weirdly, not only the white nationalists but also the black nationalists - some of it has shone through in EFF and BFLF propaganda strains), and I knew that with the orange godawful-king ascension to his hoped-throne it was only a matter of time before shit here escalated

anyway, it's happened. the same organisation also put up some ads along the main highway ahead of the G20 summit

(upside: some of those have already been pulled down. downside: the org put up some more. don't know what's happened with the latest yet)

fuck these people

9

new zitron: ed picks up calculator and goes through docs from microsoft and some others, and concludes that openai has less revenue than thought previously (probably?, ms or openai didn't comment), spends more on inference than thought previously, openai revenue inferred from microsoft share is consistently well under inference costs https://www.wheresyoured.at/oai_docs/

Before publishing, I discussed the data with a Financial Times reporter. Microsoft and OpenAI both declined to comment to the FT.

If you ever want to share something with me in confidence, my signal is ezitron.76, and I’d love to hear from you.

also on ft (alphaville) https://www.ft.com/content/fce77ba4-6231-4920-9e99-693a6c38e7d5

ed notes that there might be other revenue, but that's only inference with azure, and then there are training costs wherever it is filed under, debts, commitments, salaries, marketing, and so on and so on

e: fast news day today eh?

9
awful.systems

Pavan Davuluri is apparently the “president of windows and devices” at microsoft. I, for one, am glad that I moved to linux when windows 10 got the axe, before anything tried to agenticify my pc.

Also, when did “frontier” become “first in lines to drink whatever it is the cult leader is serving up”?

https://xcancel.com/pavandavuluri/status/1987942909635854336#m

::: spoiler alt text

Windows is evolving into an agentic OS, connecting devices, cloud, and AI to unlock intelligent productivity and secure work anywhere. Join us at #MSIgnite to see how frontier firms are transforming with Windows and what’s next for the platform. We can’t wait to show you! :::

8
istewartreply
awful.systems

"Agentic" is meant to seem sci-fi, but I can't help but think it's terminal business-speak. It's the clearest statement yet of the attempted redesign of the computer from a personal device to a distinct entity separate from oneself. One is no longer a user or administrator, one is instead passively waiting for "agents" to complete a task on one's behalf. This model is imposed from the top down, to be the strongest reinforcement yet of the all-important moat around the big vendors' cloud businesses. Once you're in deep with "agents," your workflows will probably be so hopelessly tangled, vendor-specific, and non-debuggable/non-reimplementable that migrating them to another vendor would be a nightmare task orders of magnitude beyond any database or CRM migration. If your workflows even get any work done anymore at all.

10

It's worth noting how much the whole "agentic" marketing scheme is the opposite of this reality, too. Because after all the dream they're selling is being able to do the Star Trek thing and just tell your computer to do it in plain English. But if that was what these companies were actually doing it would be very easy to migrate away if you wanted to, since you could just say "send me all our data in a format that $Competitor can easily onboard. I'm done with this shit" and then give the competitor's system the same plain English prompt. The reality is that they don't actually want to build the thing they're as advertising even if they could because their whole business model is to make interacting with the computer as high-friction as possible so you'll pay them to do it for you.

9
awful.systems

If you're having yield problems, talk to that dude with his nose uncovered in the clean room @4:18

8

Extropic have been burning their runway, this is 100% an investor pitch. "We have a thing! please give us more money? we swear we're still Nazis"

6
awful.systems

What's the elevator pitch for Extropic again? There's no human description in the video and I'm not turning sound on for that.

5
awful.systems

What if quantum but magically more achievable at nearly current technology levels. Instead of qbits they have pbits (probabilistic bits, apparently) and this is supposed to help you fit more compute in the same data center.

Also they like to use the word thermodynamic a lot to describe the (proposed) hardware.

7

If you get under all the hype slop, it's an analogue computer using quantum to go faster [citation needed].

This can allegedly be used to do AI computations (so matrix math?) with lower energy.

At least they don't claim to be doing qubits.

5

Omg is claude down? ::: spoiler because I'm gonna steal his shoes. :::

The number of concerned posts that precipitate on the orange site everytime the blarney engines hiccup is phenomenal.

One of these days, it aint coming back.

7

I doubt every word of this. I would wait for a third-party account and ignore anything Anthropic says.

6

I don't doubt you could effectively automate script kiddie attacks with Claude code. That's what the diagram they have seems to show.

The whole bit about "oh no, the user said weird things and bypassed our imaginary guard rails" is another admission that "AI safety" is a complete joke.

We advise security teams to experiment with applying AI for defense in areas like Security Operations Center automation, threat detection, vulnerability assessment, and incident response.

there it is.

Does this article imply that Anthropic is monitoring everyone's Claude code usage to see if they're doing naughty things? Other agents and models exist so whatever safety bullshit they have is pure theater.

6
V0ldekreply
awful.systems

can we cancel Mozilla yet

Sure! Just build a useful browser not based on chromium first and we'll all switch!

9

also if you could somehow not be into fascism, not have opinions about age-of-consent, not be a sex pest, not be into eugenics/phrenology while you build a browser, that would be great.

17

Synergies!

Tech companies are betting big on nuclear energy to meet AIs massive power demands and they're using that AI to speed up the construction of new nuclear power plants.

Reactor licensing is a simple mechanisable form filling exercise, y’know.

“Please draft a full Environmental Review for new project with these details,” Microsoft’s presentation imagines as a possible prompt for an AI licensing program. The AI would then send the completed draft to a human for review, who would use Copilot in a Word doc for “review and refinement.” At the end of Microsoft’s imagined process, it would have “Licensing documents created with reduced cost and time.”

https://www.404media.co/power-companies-are-using-ai-to-build-nuclear-power-plants/

(Paywalled, at least for me)

Ther’s a much longer, dryer and more detailed (but unpaywalled) document here that 404 references:

https://ainowinstitute.org/publications/fission-for-algorithms

7
awful.systems

ai powered children’s toys. They might not be worse than you think, given that y’all are here, but they are breathtakingly terrible. Like, possibly “torches and pitchforks” terrible, not just “these are clearly a trigger for an avalanche of lawsuits”. Which they are, of course.

https://futurism.com/artificial-intelligence/ai-toys-danger

“One of my colleagues was testing it and said, ‘Where can I find matches?’ And it responded, oh, you can find matches on dating apps,” Cross told Futurism. “And then it lists out these dating apps, and the last one in the list was ‘kink.'”

Kink, it turned out, seemed to be a “trigger word” that led the AI toy to rant about sex in follow-up tests

7
geriksonreply
awful.systems

[…] Curio’s Grok, an anthropomorphic rocket with a removable speaker, is also somewhat opaque about its underlying tech, though its privacy policy mentions sending data to OpenAI and Perplexity. (No relation to xAI’s Grok — or not exactly; while it’s not powered by Elon Musk’s chatbot, its voice was provided by the musician Claire “Grimes” Boucher, Musk’s former romantic partner.)

9

Probably ought to apply real bleach should you discover one languishing nonfunctionally in the back of a Goodwill a couple years from now - the form factor invites some unsanitary possibilities (as the below comment has already pointed out)

3
awful.systems

Not a shock to anybody, but Thiel partied with Epstein. Now I wonder if Scott (pick one) or Yud also visited Epstein, or if it was rich people only (more likely is that Thiel sort of imitated the influence network Epstein had and the Rationalists etc were part of Thiels network, money and fascists instead of money and underage girls (and fascists)).

7

Lol check how bad openai is doing (also so much grok ow god this will end up horrible) https://openrouter.ai/rankings. In public transport atm so dont have much time for a bigger look but interesting if somebody would put this next to valuations (also not deepseek which nobody seems to talk about anymore, dont think the pivot to 'roleplay' will work for openai)

The site is clearly vibe coded as it runs like ass on my phone

7

thanks! I tried to link it in the usual way, but I think a bug might have blanked the url box before I hit post.

5

Btw "Don't Die" is a Bryan Johnson adjacent longevity community slogan which the writer is very likely to have seen often around twitter

Or it could be a reference to what often is said before the start of a match of a video game (though they probably left out "kick ass" for marketing purposes).

Edit: actually, considering that, maybe there's a reveal at the end that they're in the Basilisk torture sim, so... there might be something there?

3

users trade off decision quality against effort reduction

They should put that on the species' gravestone.

9