Why I Resist Generative “AI” by Esther Weidauer
I was surprised to see that this lovely write-up wasn't posted here already. Also see the author's Mastodon post about it: https://wandering.shop/@aesthr/117470082684445346
https://estherweidauer.com/articles/why-resist-generative-ai.htmlOpen linkView original on lemmy.world
23 replies
@dont >>“AI is here to stay”
OK. So is heroin. And I wouldn’t recommend that to anyone either.
But seriously, this really isn’t an argument for or against anything. No technology fully vanishes again once it has been developed. That doesn’t mean though that it remains widely adopted, nor that it’s even a net good.<<
Let's make "AI" the laserdisc of our time.
It really is worth a read. Thank you.
That was a really interesting and well-written read, though I got a bit lost when the logic took some sharp turns. The idea of fascism in LLM systems was new to me and certainly got me thinking.
Because it'll burn the planet?
Amazing article. Thanks for sharing.
"AI" is certainly environmentally damaging and dangerous, and if it isn't given new input, it quickly starts generating absolute crap, but I don't agree with this statement:
We've seen new mathematical proofs coming out of LLMs. Those proofs weren't in the dataset that they were trained on.
AlphaGo Zero was given the rules of go and made to play against itself with no example games. It came up with new openings that hadn't been seen before, and were better than anything that humans had come up with.
AlphaFold was given protein sequences and was able to generate correct 3D structures, that again weren't in its training data.
Trying to defend the position of "nothing new comes out of AI" will necessarily lead you to such a narrow definition of novelty, that it will exclude most things that we consider to be human creativity.
Not the author of the blog post, but let me adress that anyways.
I agree that for some purposes, what generative "AI" produces is functionally the same as a novelty. And tautologically, even random output of a machine can contain data that humans may interpret as a "new thought" or "new knowledge" of some kind. But "Knowledge" is an intrinsically social construct and no technology can enhance it on its own, in the greater scheme of things.
Re the first point, speaking as a mathematician: The proofs generated by "AI" thus far are mostly hardly comprehensible and on their own don't actually enhance our knowledge in a meaningful way. We first have to dig through the mud. Only because 15 million dollars or whatever worth of tokens could synthesise a smooth blow up of Navier--Stokes doesn't mean we've actually learned something new and as fas as I've been told (or as far as people have made sense of the mechanics) there were no genuinly new ideas involved.
Re AlphaGo: Not in the scope of "AI" as defined in the article. Given a strict ruleset and given enough computational power, sure you can train a neural network to develop new strategies by just pure chance. And it will "learn" what a good strategy is by objective criteria like winning in simulated games etc.
Human creativity is not devalued by saying that "LLMs are simply not capable of producing new knowledge" because knowledge exists only in our minds.
Neither AlphaGo, not AlphaFold are LLMs though.
Also the mathematical proofs you're talking about seem to be mostly stolen by OpenAI from scientists that interacted with their chatbot before publishing their research.
That not fully true. The “problem” with math is that it is vast, so the building blocks for a proof are available but there is hardly anyone that has a wide enough interest to know about all the components. But AI is great at that. It read all the papers in all the disciplines and now can chain reasoning spanning all of it, a feat that’s impossible for a human.
And this makes their advances very tricky to evaluate. If a human did that, they would be a genius, but how do you evaluate a computer that did a very fancy word search across all publications?
Computers have "come up with" solutions to problems that eluded human-only problem-solving for decades. That's literally what computers are supposed to be for. That proves nothing about the novelty of AI beyond a narrow range of incremental problem-based solving which, again, we've used computers for for decades.
Characterizing previous computer use in creating proofs as the same thing as current ai usage is pretty wild
So you don’t understand how the technology you’re defending functions then?
You see that as "defending" the technology?
That specifically is not the part I was referring to, but for the record, if an LLM isn’t given any input, it does nothing.
Giving it input is how you “prompt” it to generate the absolute crap.
I was referring to the part where you gave the LLMs credit for solving math equations when there is evidence that they simply stole the solutions from real mathematicians and attempted to pass it off as work the LLM had done itself.
All maths is based on previous work. The QRH proof combined known maths in a way that nobody had thought of before.
I’m not familiar with the equation, but if you’ve got something you think is worth reading I’d take a look at it
If it doesn’t have an actual human mathematician who has checked its work and verified it, then I don’t care
Do you disagree that the examples given are new knowledge? I'm sure OP is aware AI is "just a token generator" or whatever; that doesn't really matter here when there is proof of it at least producing data in the convex hull of its training data, and in the case of alphago, with no training data at all.
They literally stole the answers to those math problems from human mathematicians who hadn’t published their work yet, but had spoken to a chat bot about it.
All 300 of them?
Probably, but I haven’t researched all of them.
The first one they claimed has been verified to be 100% stolen
I’m assuming they released the remaining 300 in one large batch so it would be more difficult to check. Like a Gish Gallup.
https://en.wikipedia.org/wiki/Gish_gallop
I think the scenario in which they steal a solution and then release 300 more stolen solutions to cover it up is highly unlikely. Gish galloping doesn't work when the news cycle has weeks to cover what you've published, if literally everything was stolen, we would've heard something by now.
But that isn't how the news cycle works.
The news cycle lasts for maybe a week. Tops.
You don’t think they’d be bragging about how it got all of them right by now if that were the case?
If they had 301 new solutions, why would they release the stolen one first? I suppose it could be because it’s the only one they were confident was correct, but I think these people are likely hubristic and psychotic enough to believe that their plausibility engine actually solve all of the unsolved math questions (rather than providing plausible looking answers, as it’s explicitly designed to do, which then need to be disproven by actual thinking human mathematicians).