Cory Doctorow: The people who tell you ‘AI is changing everything’ are lying
https://www.thenerve.news/p/cory-doctorow-ai-business-incentive-to-lie-useless-chatbots-report-nikhil-sureshOpen linkView original on lemmy.caSyndicated from the fediverse. Read and engage on the original instance.
View original on lemmy.cahttps://www.thenerve.news/p/cory-doctorow-ai-business-incentive-to-lie-useless-chatbots-report-nikhil-sureshOpen linkView original on lemmy.ca
68 replies
Sounds familiar. The proof-of-work blockchain scheme which was the fad a while ago is also often described as a solution in search of a problem.
This is stupid. The tools are helpful. Overhyped? Sure. But to pretend they are in no way helpful is just wrong.
Nothing is helpful until you need help. The fact that you think AI is helpful for you don't have to force me to use AI. If you pay me $100k+ per year and you force me to use your tools to do my job why you even hired me ? To force this cult on me or to do the job ?
I mean they are changing everything because they are being pushed and forced to be used even if they are making jobs and things in general worse and harder.
I was talking about roof sheathing with the latest chat gpt.
It was confident the joists should go above the sheathing.
You might want a 2nd opinion, did you ask Grok?
I work in construction and so far I must admit that I have no idea what the hype is about.
But I have decades of experience, meaning I don't know shit about anything BUT I know how to quickly find usefull information in the codes or in pertinents books.
If I were younger I might have fallen into the trap. If so I would probably be in jail by now, because in my tests I've seen LLM be dangerously wrong.
LLMs seem to be pretty bad at home improvement sort of questions in my experience. I suspect they don’t have a team of carpenters they use to RL the models, like they do for software engineers, etc
Software is easier because you can basically 'inspect the house' and the 'home inspector' via software.
Harder to do that with a house. Also a lot of different rules based on locality.
Reminder it doesn't know anything.
It is just mimicking text it has been programmed for.
For it to be "good" at mimicking home improvement, it's source needed to be full of home improvement talk. The best carpenters, plumbers, whatever aren't on reddit, or wherever talking shop. They're working, or making OSHA jokes at the bar.
It has already changed a lot of things in my life to the worst. that I am scared what it would look like once It changes 'Everything'
I don’t think they’re all lying; most are just misinformed or using the term differently than Cory.
It’s like saying “global warming is changing everything.” Technically, not true, but the knock on effects of specific aspects of it recently hit an inflection point that causes it to affect the lives of everyone.
This has also happened in the field of artificial intelligence; LLM chatbots are only the visible mushroom fruits of the vast mycelium network that has been silently growing underground for years.
And not so silently, for decades. Even J. Edgar Hoover was using early forms of AI to manage his data trove.
I generally respect this guy's opinion (I probably use the term "enshitification" multiple times a week!), but pretending that "AI" and "Chabot" are interchangeable terms seems wildly reductive, and the subtle presumption that the process of changing is a binary changed/non-changed situation instead of a curve is just ignorant.
If the internet was "turned off" after 4 or so years not much would have changed, either.
That's a questionable statement that may reflect on your main argument. After all, when was the Internet "switched on"? Sounds very binary to me.
To be clear, Doctorow discussed switching off AI and proposed that there would be little to no change on the world. I was just changing the technology to show the flaw in the stance.
Corey Doctorow isn't speaking to you, he's speaking to the many, many people who have been fooled by the ai craze.
And often I wonder, how many people have been fooled, and how much of the enthusiasm and over the top fantasy is astroturfing by a few people incentivized to sway public opinion?
To the average person (and this includes politicians, basically anyone not a software engineer developing the tools )who Cory generally writes to try and inform, they are the same.
Should they be? No. But that's a whole different thing and trying to change that in the general public mind doesn't change his current point about AI as the public thinks of it right now.
Speaking at the level of your target audience is wise, but there is a point where information is not just simplfied, but lost, and I think equating chat bots with AI in the broader sense is past that point. It would be like equating websites with the internet.
I think the comparison is acceptable with maybe a terminology clarification in the footnotes, because "AI" is the false term under which this current bullshit is being marketed.
And people who believe the core mechanism in large language models to be AI are so uneducated that they will probably neither understand the distinction to legitimate AI research, nor bother to read footnotes.
That's exactly the thing. Those who understand such a footnote don't need it, and those who might learn more from it are unlikely to understand it. The above commenters are simply being pedantic.
Is equating chatbots with AI any different than equating a bunch of if/else statements with AI? Is any of this AI considering none of it involves actual intelligence or informed decision making?
Sure AI is more than LLMs, but those are the face of it in the public's eye and is what's driving these trillion dollar corporate valuations, which then lead to every company on earth declaring that their product is "AI!"
As a non-techie: websites are the internet.
Like, I conceptualize the internet as basically just a giant shared network, but the point of it is the websites (for me).
I apologize to anyone I may have just gravely offended.
lol
I don't know how "non-techie" you are, but a good example is online games. When you're playing Fortnite, Roblox, World of Warcraft, etc you're not going to a web page, but you're on the internet.
I straight up almost called it a series of tubes, lol.
Yeah, that’s probably why I do think of it as a shared network, I just don’t play online games, so that’s not what I associate with the internet.
There are other examples but I take your point.
What's lost? I can't think of anything.
Words are meant to convey meaning, and I bet you would lose a lot of non-technical people if you tried to explain the intricacies of what makes large language models worse than content recommendation systems, and doing this would be redundant for technical users.
So Corey can safely use the two words interchangeably and communicate with technical and non-technical alike.
then he has completely failed to understand the concepts for people using him an a information proxy
isnt he supposed to help inform? or is he just an echo chamber to the lowest-common-loudmouth?
Stepping up and doing it better is free.
A lot of people don't use it the way he used it though. He used it specifically for two sided markets (eg a service that has both business and personal users) where the focus shifts to the business customers, but people have started using it to mean anything that used to be good but isn't good any more.
I think you're making a strawman argument here. He isn't arguing that it's a binary switch that has failed to throw, he's arguing that we don't have compelling evidence that the ends justify the means:
And this is completely true! Supposedly "data informed" organizations are trying to find a yardstick that shows actual, meaningful improvement in business outcomes from AI. In my own organization we have people touting LOC yet again because "big number", but anyone who's ever worked in software can tell you it's an asinine metric to use as a KPI.
It's inherently unscientific to start with the answer and work backwards to a satisfactory question. His point that this push is coming from the least knowledgeable of real processes- and more closely resembles religious fervor than business acumen- seems to at least warrant consideration.
He is definitely pretending that change is either on or off.
Are CEOs jumping the gun on how quickly they adopt AI into workflows? Definitely. However, there's a big difference between "AI isn't at a threshold where it is disruptive" and "AI isn't disruptive". Or, to belabor the metaphor: CEOs are jumping the gun, but the race is about to start and they're on the correct track.
A key word in that sentence is "if", it is a rhetorical example, that's probably pretty much correct at this point. It's not a recommended course of action, it's a declaration that it contrary to crazed hype, it isn't currently as core to everything as would be befitting the current hype level.
I would argue that the CEOs aren't on the correct track, they aren't really in a particularly specific trajectory. I just had a debate with someone on this and their stance was "well in a hundred years do you expect things to be like they are". My response "I cannot possibly speak to that, but we need to speak to today instead of pretending we know how things will be in a hundred years and pretending they are already at that level". The "imagine a hundred years from now" by a relative outsider to the tech is dominating CEO mindset, and that's problematic.
Once the bubble pops, we will almost certainly see a more durable and sane adoption for these technologies. For now, the hype is a problem as it enables some of the worst possible stewards of the technology and favors grift over progress.
If you watch a full podcast (he has done a ton in the last couple weeks), he clearly identifies AI as not a mere hype technology, as something interesting and potentially useful from a technology perspective, and he definitely doesn't conflate a chatbot with all AI. That doesn't undermine the vast problems with it, how it's being used against regular workers, the threat the bubble poses to the economy, the criti-hype cycle, etc.
I think you're oversimplifying "Cory Doctorow" based on the article you're reading or specifically how he presents a more complex idea to different audiences.
Watch the Jon Stewart podcast interview if you want more nuance:
https://www.youtube.com/watch?v=-dAIJRjb-Bw
"Chatbot" style AI is wildly good and bad at varying kinds of tasks, and a lot of that has to do with how it has been prepared.
Some LLMs have been trained to make images - I've not been too impressed with them, but that's what they're "good" at - and better than the LLMs that have been trained to write computer code when you ask the coding LLMs to draw a picture.
The code writing LLMs have actually improved the most at reviewing code over the past 8-9 months, and that ability to review their own code makes them dramatically better at writing code as well.
I find Google Gemini to be pretty impressive at scanning laws and regulations and finding, not creative, but functional solutions to stated problems within the constraints of (often frustratingly bizarre) legal structures.
And all of them will lie to you, tell you what a great idea you have, etc. They're not really lying, they're mostly just taking what they read at face value without checking corroborating sources enough to find the obvious (to you) blunders. If you want the LLM to be sure, ask it to go on the RAG (Research Augmented Generation) - check everything before saying it, they can do that, especially "paid mode" engines, but it reduces their capacity for analysis of complex problems by 3-10x, because they're spending so much context window "being sure" - you can alternatively spend 3-10x as long solving complex problems / accomplishing complex tasks if you have them do their homework, verify everything from "the best" available sources 3x and build up a local document set of "trusted information" which is used in preference to whatever it might find at random on the internet. This isn't as sexy as "Hey Claude, code me up a database that does X Y Z" and getting the result in 30 seconds, but it is how professionals have been doing their jobs for centuries: learn reliable information first, then act on it.
In the same way that most people if you say you'll take them to their destination with your vehicle, they assume in your car and not on the back of your bicycle, although both are technically correct.
I think it's a worthy simplification that sums up the current state of things well. We have AI that broadly has different uses and then we have the ChatBot/ChatBot derived AI applications. The market is not going insane over the broader AI category. Executives are not tripping over themselves to say things like machine vision is going to replace all labor, or at least all white collar labor. The chatbot is the only thing in the conversation of consequence.
All the "everyone says AI is the reality, so everyone feels like they must say AI is the reality" refers to this specific category. It might be veering towards being oversimplified, but it's trying to balance a perspective that is also oversimplified. Generally, we aren't good at weighing simple straightforward takes against complex nuanced takes, so you have to "net it out" to have any hope of the point landing.
Pretending AI and Chatbot are interchangeable is a marketing strategy by the companies who make these advanced chat bots. "They are artificially intelligent bro".
I don't disagree that the term AI has shifted significantly in the last 4 years or so, but I was actually meaning it in the other direction, as in, there is more to generative AI (what people are calling "AI" these days) than just chat bots.
Oh look it's arbitrary-narrow-definition-that-fits-the-argument-I-want-to-make man!
I'm saying there's more to generative AI than chatbots. My definition of AI is broader. What are you going on about?
On the contrary - the people who tell you AI doesn’t add any value and will disappear in a few years are either lying, dumb, or completely ignorant of what AI can already do.
Read the article
I'm a software developer since the 90s, basically before the internet, we had some C books for reference and that's it. I can tell you that I started last year to use copilot in vscode and some chatgpt on a web page, and it basically changed my world, and all my 50+ years old coworkers are amazed by what it can do really.
You are right it will not fade at all in software development.
Completely disagree. The difficult part of software development was never writing code OR speed of delivery. It was understanding requirements and problem solving. LLMs still can not do either of those things and there is no evidence they ever will be able to.
An example of how harmful LLMs actually are to development can succinctly be described with an issue I had a few weeks ago. I found an issue in an open source project, code was fine if a bit hard to understand. I came up with a PR to fix the problem.
In the time from me checking out the code to submitting the PR, a little less than 24 hours, the maintainer had completely rewritten the entire project with Claude. It was complete nonsense. Incredibly difficult to understand. Abstracting things that didn’t need abstracting. My PR was useless, because the entire project was new. The maintainer definitely didn’t understand the changes either. If a bug came up there’s no way AI would be able to solve it (the bug was still there even though the code was entirely new).
LLMs don’t understand the code. They just make things that look like they will work. And then a human has to maintain it (or keep paying billions of dollars for Claude to try to fix it).
the person in your scenario is just fkin stupid. When people say that llms are helpful for coding, they aren't talking about telling an llm to rewrite an entire codebase.
You are right, but right now we are living in a very messy reality where it's hard to know who's being stupid and who is using it well. One person I know that, well, I was never a huge fan of his work but at least it was somewhat serviceable is now all-in on AI and his code has been rewritten in a similar manner as the parent poster comments. He's got no idea how it works or how it should worked, the AI decided to rewrite it in an entirely different language, and it's a buggy mess and it never fixes the bugs without making new bugs. Then he hit his token quota 3 weeks early and basically said he was going to stop working on it because he no longer could manually work the codebase. He didn't ask for a rewrite, but AI advised him that his language choice was a poor fit and reworked things in another language, one that none of us use to that level of seriousness. It also made it largely based on super convoluted regular expressions.
The problem is that the leadership is singing the praises of these people, they were on their 'leaderboards' of AI adoption and much like the craze of praising "lines of codes", we are neck deep in the most stupid non-technical evaluation of technical work you could imagine.
In other words, you're arguing that their usage without discernment is detrimental.
Cory agrees on this very same article, and it also includes the same nuance this chain is trying to give voice to:
We wouldn't be having this conversation if LLMs had been given the chance to grow into being the same way the web did. Whereas we would be having this same conversation with the letters swapped if corporations were the ones to spawn the WWW instead of the way it came about.
The reason is the same. There is only one war.
It's not just "usage without discernment", it's that AI flattens some costs which are very obvious and very measurable (coding) in such a way that it introduces or amplifies other costs which are much more diffused and hard to measure (code and design reviewing, bug fixing, maintenance, adding new requirements), plus AI is totally incapable of doing the higher level tasks that shape what code needs to be done (technical analysis, requirements analysis and in bigger companies technical architecture).
People who are non-experts, aren't really senior domain experts or have some kinds of unbalanced expertise (they've never really progressed beyond being a coder, or they don't have full life-cycle experience with big projects or they're in an industry or position where they just make the code, shove it out the door and it's not their problem anymore) just look at the one thing they in their ignorance think is THE cost in programming - coding - and go "hey, this AI thing is amazing" even while AI is creating all sorts of much more time consuming problems which they don't really understand formally (they think those things are just "bad luck", "there's nothing we can do about avoiding this" and "it's just the way things are in programming") that require the time of people with higher expertise levels (i.e. who are more costly) to solve and AI doesn't even help with the kind of stuff which if done wrongly or not at all can condemn a software project to fail before it even starts like just half-way decent Technical and Requirements Analysis.
That's why you get some programmers going "this AI shit is amazing" whilst the really senior software development types are just nodding their heads and thinking "these people are ignorant as fuck juniors".
I'm sorry but to me your comment is a bit misguided. You raise extremely valid point and are completely right in what you say, and yet all your argument fails to prove that software development hasn't changed.
The difficult part was understanding requirements and problem solving: absolutely true. Yet most of the time of a developer was spent in writing code. Now it's spent refining the analysis so that the LLM stops producing slop. And many programmers are doing it, even with all its downsides, because for them the fun part is understanding the requirements and problem solving, not writing code nor delivering fast. They are delegating those tasks to a machine, even with all the risks and issues.
Your second point (and the anecdote) further proves how programming changed. Before it was unthinkable that some random person, likely with no clue about what they are doing, would refactor an entire codebase in a night.
Both are massive changes. For the best? Arguably not, but I seriously doubt there will be any going back now.
The thing is that actually doing this isn't faster than writing the code, robs the practitioner of learning, and more often than not doesn't actually happen, so you have a harder to maintain codebase with more bugs and less knowledgeable developers to maintain it.
Edit: and as a fun bonus accelerates glacier melting!
I think you're right in general. I think juniors and those who havent yet had experience are not going to understand a goddamn thing and produce broken, insecure, unmaintainable slop.
I've written my share of garbage code -- completely by hand! And i'm much better for it.
Once you have that experience, once you've written a few backends and frontends, there's not much left to understand. Move the data from here to there. Display it, transform it, slice it up. For webdev AI is a huge force multiplier. I can make a dozen features or apps in the time it used to take me to learn one framework I was curious about. It even helps me learn faster because of how quickly I can test new patterns and ideas.
There's certainly a right way to use it if you want to continue being edified, burning the planet down aside.
If we only used AI for codegen, this probably wouldn’t be much of an issue. Those cat videos take more energy than building a complete app. Also, it’s all pretty new and the newest tech 40 years ago would have filled a warehouse and had the computational power of a potato, but here we are now. I expect we’ll get more efficient at it and in the ways we use it. And there’s already a huge worldwide shift in energy capture (America aside…)
Whenever I look back at old code, mine or others, the first words that usually come to mind are "what you have to understand about this is... we were on a tight schedule, we never thought this was going to be used in production, we weren't allowed to execute the planned and contracted refactor... etc. etc. etc."
Fully agree.
Unfortunately many people would rather spin the wheel for a chance to win magically produced functioning code, rather than doing the work themselves with sure results - even if it takes the same or more time. And the kind of current politicians there are around the world proves that most people don't give a fuck about the ice-caps (though I would also argue that it's not so much the random person calling an LLM that is poisoning the waters - even though it does have a non-negligible effect -, rather it's massive sociopaths in charge of the companies creating LLMs that are perfectly fine with destroying the environment and other people's money in a vain dream of being the owner of some kind of "new order")
Not at all. It gives the practitioner the option of skipping the learning.
Starting in the 1990s I started skipping the learning of assembly language, compilers got good enough that I just don't need to know how the latest SIMD/MIMD/ whatever instructions work, I just express what I want in C and gcc or whatever handles the optimization for me.
Comparing it to an (almost) entirely complete abstraction where you (almost) never have any benefit to looking under the covers like c over assembler is completely dishonest.
Is it, though? In the early 1990s I could still optimize compiler output by hand, here and there. In the 1980s it was common practice and necessary in many circumstances to make complex things happen on the constrained hardware. In the 1970s there were a lot of programmers who never touched Fortran, just practiced assembly all the time because Fortran was too inefficient for their needs.
I'll say that LLMs, this year, are something like compilers were in the 1960s - a revolutionary improvement in accessibility of coding, being able to express what you want in "natural language" - like COBOL did starting in 1959.
LLMs have plenty of pitfalls that COBOL doesn't today, but I'll note that Borland Turbo C++ compiler in 1991 was too damn buggy to do anything much more complex than "Hello, World." with.
This is a point that really sticks with me. Using it for the sometimes spot on cakewalk segments is a fairly productive win. By the time you stubbornly insist on driving it entirely blackbox with chat and trying to get the right results without actually touching code... Well, even when it works, it's often more work than just doing it yourself.
Someone rebased a UI I worked on in a new version of the UI framework. As a result, there was this one odd gap in the UI in one specific place. A vibe coder spent 3 hours back and forth with the AI trying to get it to correct the gap and finally submitted their merge request. Hundreds and hundreds of lines of CSS. So I declined the merge request, open the gui, looked at the gap, hit f12, adjusted a single padding statement, and it was all good. People are struggling with defining all sorts of criteria and rigging it to let it try and try and try again and hopefully laid out every contingency, every corner case, and spent hours laying the ground work and could have done similar in a more straightforward way.
One thing that absolutely blows my mind is how many people will say how much time it saves then with repetitive or boilerplate code. It is obvious these people have never actually tried to optimize their workflow even a little bit before. Regex replace, snippets, and keyboard macros have existed in text editors forever and are actually deterministic.
As little as possible of my work is boilerplate, but some times there will be something like "I want to take width,height, and depth as arguments from the command as variables", and poof, all the argv tedium is done.
That's what developers told the world. Now they're exposed, it never really took that long to write the code. (Only partly joking.)
Actually, a whole lot of time went into reading other developers' code, getting documentation in sync with the actual implementation. And if you didn't do all that, you tended to have a lot more bugs / vulnerabilities, etc. The LLMs are wicked fast at reviewing code, they don't find ALL the problems, a lot of problems they do find aren't worth fixing, but they do find more actual actionable problems per minute than most developers can find per hour in a big code base.
I don't know... I just made a scheduling / timesheet creation app. Multi-user, overlapping clients and providers, multiple funding sources. Took 10 calendar days to make the initial app working part time, maybe 2-3 hours a day. Initially written in Python, decided at that point I'd rather have it in Go. Because the initial app had robust requirements and design docs, the translation to Go happened in less than 5 calendar days, with almost zero human involvement beyond telling the agent "continue" at each stopping point. After the Go translation was done (and debugged by the LLM to a flawless translation - only difference is that it runs faster), I was given a new timesheet to use for some of the workers, weekly instead of bi-weekly. Pay weeks start on Monday instead of Thursday. Various wrinkles about how the employees and clients and services are identified, weird sub-totals by service. All I told the LLM was: "Here's a new timesheet that we'll be using for some workers, design the necessary modifications and extensions to accomodate it." It did, independently. It highlighted three shortcuts it took and I told it not to take those shortcuts, it adjusted.
That's not quite rocket science, but it's still impressive: to dissect the given .pdf, determine what data goes in what fields, in what formats, with what calculations, based on just reading the page, then adapt the existing app to fill it out automatically.
Tech in general. I'm a sysadmin in research computing. You know how many clients at prestigious universities are using AI for mathematics, biology, etc? All of them
I'm my experience people give tech demos, everyone is impressed, my coworkers say it makes them multiples faster, then I have more work making sure the wheels don't fall off.
I've spent the last 4 months using LLMs to review / ensure that junior coders' wheels don't fall off.