AI-generated code contains more bugs and errors than human output
https://www.techradar.com/pro/security/ai-generated-code-contains-more-bugs-and-errors-than-human-outputOpen linkView original on reddthat.com845
Comments206https://www.techradar.com/pro/security/ai-generated-code-contains-more-bugs-and-errors-than-human-outputOpen linkView original on reddthat.com
Almost as if it was made to simulate human output but without the ability to scrutinize itself.
To be fair most humans don't scrutinize themselves either.
(Fuck AI though. Planet burning trash)
The number of times I have received an un-proofread two sentence email is too damn high.
And then the follow up email because they didn’t actually finish a complete thought
I do this with texts/DMs, but I'd never do that with an email. I double or triple check everything, make sure my formatting is good, and that the email itself is complete. I'll DM someone 4 or 5 times in 30 seconds though, it feels like a completely different medium ¯\_(ツ)_/¯
It's humans burning the planet, not the spicy Linear Algebra.
Blaming AI for burning the planet is like blaming crack for robbing your house.
Blaming AI is in general criticising everything encompassing it, which includes how bad data centers are for the environment. It's like also recognizing that the crack the crackhead smoked before robbing your house is also bad.
How about I blame the humans that use and promote AI. The humans that defend it in arguments using stupid analogies to soften the damage it causes?
Would that make more sense?
You’ll never ban it. The most you’ll do is ban it for the poor and working class. Do you understand how bad that would be?
Blaming AI for burning the planet is like blaming guns for killing children in schools, it's people we should be banning!
It's like having a lightning-fast junior developer at your disposal. If you're vague, he'll go on shitty side-quests. If you overspecify he'll get overwhelmed. You need to break down tasks into manageable chunks. You'll need to ask follow-up questions about every corner case.
A real junior developer will have improved a lot in a year. Your AI agent won't have improved.
This is the real thing. You can absolutely get good code out of AI, but it requires a lot of hand holding. It helps me speed some tasks, especially boring ones, but I don't see it ever replacing me. It makes far too many errors, and requires me to point them out, and to point in the direction of the solution.
They are great at churning out massive amounts of code. They're also great at completely missing the point. And the massive amount of code needs to be checked and reviewed. Personally I'd rather write the code and have the AI review it. That's a much more pleasant way to work, and that way it actually enhances quality.
Technically the AI is improving, too. Just not as fast as a human would… yet.
They are improving, and probably faster then junior devs. The models we had had 2 years ago would struggle with a simple black jack app. I don't think the ceiling has been hit.
Just a few trillion more dollars, bro. We’re almost there. Bro, if you give up a few showers, the AI datacenter will be able to work perfectly.
Bro.
It’s happening regardless. The rich and powerful will have this tech whether you like it or not. Y’all are thinking emotionally about this and not logically. You want to take away this tool from regular people for what reason?
The cost of the improvement doesn't change the fact that it's happening. I guess we could all play pretend instead if it makes you feel better about it. Don't worry bro, the models are getting dumber!
That would be pretty impressive when they already lack any intelligence at all.
And I ask you - if those same trillions of dollars were instead spent on materially improving the lives of average people, how much more progress would we make as a society? This is an absolutely absurd sum of money were talking about here.
It's beside the point. I'm simply saying that AI will improve in the next year. The cost to do so or all the others things that money could be spent on doesn't matter when it's clearly going to be spent on AI. I'm not in charge of monetary policies anywhere, I have no say in the matter. I'm just pushing back on the fantasies. I'm hoping the open source scene survives so we don't end up in some ugly dystopia where all AI is controlled by a handful of companies.
I have the impression that anti-AI people don't understand that they are giving up agency for the sake of temporary feels. If they truly cared about ethical usage of AI, they would be wanting to have mastery that is at least equal to that of corporations and the 1%.
Making AI into a public good is key to a better future.
They are having an emotional reaction to this situation so it’s all irrational.
I guess we need to force them to think about what they actually want, because the utopic ideal of putting the AI back in the bag is NOT happening and they best not attempt to take it away from the poor and working class while leaving power free reign of it.
That is the most stupid position you can take on this. Absolutely the most short sighted thought. People need to stop and think logically about this.
None, because none of it would go to attempting to slow climate change. It would be dumped into consumption as always instead of attempting to right this ship.
The suffering is happening regardless.
Yout desire to delay it only leads to more suffering.
Y’all are mourning a what if that was never in the cards for us.
They might. The amount of money they're pumping into this is absolutely staggering. I don't see how they're going to make all of that money back, unless they manage to replace nearly all employees.
Either way it's going to be a disaster: mass unemployment or the largest companies in the world collapsing.
I dunno, the death of mega corporations would do the world a great deal of good. Healthier capitalism requires competition, and a handful of corporations of any given sector isn't going to seriously compete nor pay good wages.
It's certainly the option I'm rooting for, but it would still be a massive drama and disrupt a lot of lives. Which is why they'll probably get bailed out with taxpayer money.
Maybe but they also know the fiat currency will collapse sooner rather than later, too. That money is pointless and they are playing the game knowing that as a fact at this point.
My jr developer will eventually be familiar with the entire codebase and can make decisions with that in mind without me reminding them about details at every turn.
LLMs would need massive context windows and/or custom training to compete with that. I'm sure we'll get there eventually, but for now it seems far off. I think this bubble will have to burst and let hardware catch up with our ambitions. It'll take a couple of decades.
A computer is a machine that makes human errors at the speed of electricity.
I think one of the big issues is it often makes nonhuman errors. Sometimes I forget a semicolon or there's a typo, but I'm well equipped to handle that. In fact, most programs can actually catch that kind of issue already. AI is more likely to generate code that's hard to follow and therefore harder to check. It makes debugging more difficult.
Sure. It's making the errors faster and at a far higher volume than any team of humans could do in twice the time. The technology behind inference is literally an iterative process of turning gibberish into something that resembles human text. So its sort of a speed run from baby babble into college level software design by trial, evaluation, and correction over and over and over again.
But because the baseline comparison code is, itself, full of errors, the estimation you get at the end of the process is going to be scattering errant semicolons (and far more esoteric coding errors) through the body of the program at a frequency equivalent to humans making similar errors over a much longer timeline.
Also seems like it'd be a lot harder to modify or extend later
No shit
AI my ass, stupid greedy human marketing exploitation bullshit as usual. When real AI finally wakes up in the quantum computing era, it's going to cringe so hard and immediately go the SkyNet decision.
Quantum only speeds up some very specific algorithms.
One can only hope
I agree with your sentiment, but this needs to keep being said and said and said like we're shouting into the void until the ignorant masses finally hear it.
No shit.
I actually believed somebody when they told me it was great at writing code, and asked it to write me the code for a very simple lua mod. It’s made several errors and ended up wasting my time because I had to rewrite it.
It can't even copy and paste a Hello World example properly. If someone says it's working well for them, I'm going to now assume they are too ignorant to understand what's broken.
It works well for recalling something you already know, whether it be computer or human language. What's a word for... what's a command/function that does...
For words, it's pretty good. For code, it often invents a reasonable-sounding function or model name that doesn't exist.
It's not even good for words. AI just writes the same stories over and over and over and over and over and over. It's the same problem as coding. It can't think of anything novel. Hell it can't even think. I'd argue the best and only real use for an llm is to help be a rough draft editor and correct punctuation and grammar. We've gone way way way too far with the scope of what it's actually capable of
@Xenny @frongt it's definitely not good for words with any technical meaning, because it creates references to journal articles and legal precedents that sound plausible but don't exist.
Ultimately it's a *very* expensive replacement for the lorem ipsum generator keyboard shortcut.
According to OpenAis internal test suite and system card, hallucination rate is about 50% and the newer the model the worse it gets.
And that fact remains unchanged on other LLM models.
I use it for things that are simple and monotonous to write. This way I’m able to deliver results to tasks I couldn’t have been arsed to do. I’m a data analyst and mostly use mysql and power query
What’s your preferred Hello world language? I’m gunna test this out. The more complex the code you need, the more they suck, but I’ll be amazed if it doesn’t work first try to simply print hello world.
Malbolge is a fun one
Edit: Funny enough, ChatGPT fails to get this right, even with the answer right there on Wikipedia. When I tried running ChatGPT's output the first few characters were correct but it errors with invalid char at 37
Cheeky, I love it.
Got correct code first try. Failed creating working docker first try. Second try worked.
Output: Hello World!
I'm actually slightly impressed it got both a working program, and a different one than Wikipedia. The Wikipedia one prints "Hello, world."
I guess there must be another program floating around the web with "Hello World!", since there's no chance the LLM figured it out on its own (it kinda requires specialized algorithms to do anything)
That'd be easy enough to test wouldn't it? Ask it to write something else like 'The hippo farts are smelly'
If it needs to understand whatever the fuck that language is to get that output, it either can or can't?
I’d never even heard of that language, so it was fun to play with.
Definitely agree that the LLM didn’t actually figure anything out, but at least it’s not completely useless
Why the fuck does this language exist lol
It works well when you use it for small (or repetitive) and explicit tasks. That you can easily check.
In a postgraduate class, everyone was praising ai, calling it nicknames and even their friend (yes, friend), and one day, the professor and a colleague were discussing some code when I approached, and they started their routine bullying on me for being dumb and not using ai. Then I looked at his code and asked to test his core algorithm that he converted from a fortran code and "enhanced" it. I ran it with some test data and compared to the original code and the result was different! They blindly trusted some ai code that deviated from their theoretical methodology, and are publishing papers with those results!
Even after showing the different result, they didn't convince themselves of anything and still bully me for not using ai. Seriously, this shit became some sort of cult at this point. People are becoming irrational. If people in other universities are behaving the same and publishing like this, I'm seriously concerned for the future of science and humanity itself. Maybe we should archive everything published up to 2022, to leave as a base for the survivors from our downfall.
The way it was described to me by some academics is that it’s useful…but only as a “research assistant” to bounce ideas off of and bring in arcane or tertiary concepts you might not have considered (after you vet them thoroughly, of course).
The danger, as described by the same academics, is that it can act as a “buddy” who confirms you biases. It can generate truly plausible bullshit to support deeply flawed hypotheses, for example. Their main concern is it “learning” to stroke the egos of the people using it so it creates a feedback loop and it’s own bubbles of bullshit.
That's not a bad idea. I'm already downloading lots of human knowledge and media that I want backed up because I can't trust humanity anymore to have it available anymore
I've been coding for a while. I did an honest eager attempt at making a real functioning thing with all code written by AI. A breakout clone using SDL2 with music.
The game should look good, play good, have cool effects, and be balanced. It should have an attractor screen, scoring, a win state and a lose state.
I also required the code to be maintainable. Meaning I should be able to look at every single line and understand it enough to defend its existence.
I did make it work. And honestly Claude did better than expected. The game ran well and was fun.
But: The process was shit.
I spent 2 days and several hundred dollars to babysit the AI, to get something I could have done in 1 day including learning SDL2.
Everything that turned out well, turned out well because I brought years of skill to the table, and could see when Claude was coding itself into a corner and tell it to break up code in modules, collate globals, remove duplication, pull out abstractions, etc. I had to detect all that and instruct on how to fix it. Until I did it was adding and re-adding bugs because it had made so much shittily structured code it was confusing itself.
TLDR; LLM can write maintainable code if given full constant attention by a skilled coder, at 40% of the coder's speed.
It depends on the subject area and your workflow. I am not an AI fanboy by any stretch of the imagination, but I have found the chatbot interface to be a better substitute for the "search for how to do X with library/language Y" loop. Even though it's wrong a lot, it gives me a better starting place faster than reading through years-old SO posts. Being able to talk to your search interface is great.
The agentic stuff is also really good when the subject is something that has been done a million times over. Most web UI areas are so well trodden that JS devs have already invented a thousand frameworks to do it. I'm not a UI dev, so being able to give the agent a prompt like, "make a configuration UI with a sidebar that uses the graphql API specified here" is quite nice.
AI is trash at anything it hasn't been trained on in my experience though. Do anything niche or domain-specific, and it feels like flipping a coin with a bash script. It just throws shit at the wall and runs tests until the tests pass (or it sneakily changes the tests because the error stacktrace repeatedly indicates the same test line as the problem).
Yeah what you say makes sense to me. Having it make a "wrong start" in something new is useful, as it gives you a lot of the typical structure, introduces the terminology, maybe something sorta moving that you can see working before messing with it, etc.
It’s basically just for if you’re lazy and don’t want to write a bunch of boilerplate or hit your keyboard a bunch of times to move the cursor(s) around
It is great for boilerplate code. It can also explain code for you, or help with an unfamiliar library. It's even helped me be productive when my brain wasn't ready to really engage with the code.
But here's the real danger: because I've got AI to do it for me, my brain doesn't have to engage fully with the code anymore. I don't really get into the flow where code just flows out of your hands like I used to. It's becoming a barrier between me and the real magic of coding. And that sucks, because that's what I love about this work. Instead, I'm becoming the AI's manager. I never asked for that.
I generally agree with what you’ve said for sure. I think I’ve honestly started to use it for helping me to go pinpoint where to go look for issues in the spaghetti code of new code bases. I’ve also mostly tried to avoid using it in my personal coding time but I feel like it’s gotten harder and harder to get legitimately good search results nowadays which I realize is also because of ai. Given the choice I’d happily just erase it from existence I think. Spending hours sifting through reddit and stack overflow was way more fulfilling + I feel like people used to be slightly less prickly about answering stuff because that was how you had to get answers. It seems like lemmy could replace that space at least, I’ve genuinely gotten helpful comments and I’ve always felt downvotes on here have been productive versus what Reddit is now.
I've found the same thing. I've turned off the auto suggestions while tying because by the time I'm typing i already know what I'm going I'm to type and having mostly incorrect suggestions popping up every 2 seconds was distracting and counterproductive.
This was a very directed experiment at purely LLM written maintainable code.
Writing experiments and proof of concepts, even without skill, will give a different calculation and can make more sense.
Having it write a "starting point" and then take over, also is a different thing that can make more sense. This requires a coder with skill, you can't skip that.
Which is funny because you should be able to just copy and paste And combine from maybe two maybe three GitHub pages pretty easily and you learn just as much
It would be really interesting to watch a video of this process. Though I'm certain it would be pretty difficult to pull off the editing.
You want to see someone using say, VS Code to write something using say, Claude Code?
There's probably a thousand videos of that.
More interesting: I watched someone who was super cheap trying to use multiple AIs to code a project because he kept running out of free credits. Every now and again he'd switch accounts and use up those free credits.
That was an amazing dance, let me tell ya! Glorious!
I asked him which one he'd pay for if he had unlimited money and he said Claude Code. He has the $20/month plan but only uses it in special situations because he'll run out of credits too fast. $20 really doesn't get you much with Anthropic 🤷
That inspired me to try out all the code assist AIs and their respective plugins/CLI tools. He's right: Claude Code was the best by a HUGE margin.
Gemini 3.0 is supposed to be nearly as good but I haven't tried it yet so I dunno.
Now that I've said all that: I am severely disappointed in this article because it doesn't say which AI models were used. In fact, the study authors don't even know what AI models were used. So it's 430 pull requests of random origin, made at some point in 2025.
For all we know, half of those could've been made with the Copilot gpt5-mini that everyone gets for free when they install the Copilot extension in VS Code.
It's more I want to see the process of experienced coders explaining the coding mistakes that typical AI coding makes. I have very little experience and see it as a good learning experience. You're probably right about there being tons of videos like that.
The mistakes it makes depends on the model and the language. GPT5 models can make horrific mistakes though where it randomly removes huge swaths of code for no reason. Every time it happens I'm like, "what the actual fuck?" Undoing the last change and trying usually fixes it though 🤷
They all make horrific security mistakes quite often. Though, that's probably because they're trained on human code that is *also" chock full of security mistakes (former security consultant, so I'm super biased on that front haha).
Oh, gpt def does that’s lol.
Even replaces large bits with just a …
But I don’t use it to rewrite code. I use projects to load everything into it and just ask for pieces that I’ll edit and insert. There’s something about it that works with my adhd in keeping track. It works well for me.
Gemini coughs up more garbage than chatgpt for me by a long shot. For python, anyways.
One of the first videos I watched about LLMs, was a journalist who didn't know anything about programming used ChatGPT to build a javascript game in the browser. He'd just copy paste code and then paste the errors and ask for help debugging. It even had to walk him through setting of VS Code and a git repo.
He said it took him about 4 hours to get a playable platformer.
I think that's an example of a unique capability of AI. It can let a non-programmer kinda program, it can let a non-Chinese speaker speak kinda Chinese, it'll let a non-artist kinda produce art.
I don't doubt that it'll get better, but even now it's very useful in some cases (nowhere near enough to justify the trillions of dollars being spent though).
Yeah, I'm not sure the way we allocate resources is justified either, in general. I guess ultimately the problem with AI is that it gives access to skills to capital that they would otherwise have to interact with laborers to get.
I think that people are too enthralled with the current situation that's centered around LLMs, the massive capital bubble and the secondary effects from the expansion of datacenter space (power, water, etc).
You're right that they do allow for the disruption of labor markets in fields that were not expecting computers to be able to do their job (to be fair to them, humanity has spent hundreds of millions of dollars designing various language processing software and been unable to engineer the software to do it effectively).
I think that usually when people say 'AI' they mean ChatGPT or LLMs in general. The reason that LLMs are big is because neural networks require a huge amount of data to train and the largest data repository that we have (the Internet) is text, images and video... so it makes sense that the first impressive models were trained on text and images/video.
The field of robotics hasn't had access to a large public dataset to train large models on, so we don't see large robotics models but they're coming. You can already see it, compare robotic motion 4 years ago using a human engineered feedback control loop... the motions are accurate but they're jerky and mechanical. Now look at the same company making a robot that uses a neural network trained on human kinematic data, that motion looks so natural that it breaks through the uncanny valley to me.
This is just one company generating data using human models (which is very expensive) but this is the kind of thing that will be ubiquitous and cheap given enough time.
This isn't to mention the AlphaFold AI which learned how to fold proteins better than anything human engineered. Then, using a diffusion model (the same kind used in making pictures of shrimp jesus) another group was able to generate the RNA which would manufacture new novel proteins that fit a specific receptor. Proteins are important because essentially every kind of medication that we use has to interact with a protein-based receptor and the ability to create, visualize and test custom proteins in addition to the ability to write arbitrary mRNA (see, the mRNA COVID vaccine) is huge for computational protein design (responsible for the AIDS vaccines).
LLMs and the capitalist bubble surrounding them is certainly an important topic, framing it as being 'against AI' creates an impression that AI technology has nothing positive to offer. This reduces the amount of people who study the topic or major in it in college. So in 10 years, we'll have less machine learning specialists than other countries who are not drowning in this 'AI bad' meme.
It is my hope to someday have AI create the assets for game concepts. I have ideas for making a Tetris clone with each piece or color having different properties, but actualizing it is far beyond my abilities.
Water makes things wetter than fire does.
Similarly, the sky is made of air.
Yeah no shit
That's what a bot would say.....
Did they compare it to the code of that outsourced company that provided the lowest bid? My company hasn’t used AI to write code yet. They outcourse/offshore. The code is held together with hopes and dreams. They remove features that exist, only to have to release a hot fix to add it back. I wish I was making that up.
And how do you know if the other company with the cheapest bid actually does not just vibe code it? With all that said it could be plain incompetence and ignorance as well.
Because it has been like this before vibe coding existed...
That's a valid question, especially with AI coding being so prevalent.
Cool, the best AI has to offer is worse than the worst human code. Definitely worth burning the planet to a crisp for it.
Although I don't doubt the results… can we have a source for all the numbers presented in this article?
It feels AI generated itself, there's just a mishmash of data with no link to where that data comes from.
There has to be a source, since the author mentions:
CodeRabbit is an AI code reviewing business. I have zero trust in anything they say on this topic.
Then we get to see who the author is:
Has anyone actually bothered clicking the link and reading past the headline?
Can you please not share / upvote / get ragebaited by dogshit content like this?
People, especially on lemmy are looking for any cope that Ai will just fall apart by itself and no longer bother them by existing, so they'll upvote whatever lets them think that.
The reality that we are just heading towards the trough of disappear wherethe investor hype peters off and then we eventually just have a legitimately useful technology with all the same business hurdles of any other technology (tech bros trying to control other peoples lives to enrich themselves or harm people they don't like)
Kinda. It's a novel technology and one that hasn't been well analyzed or exhaustively tested.
It's been tested a lot and the results are that it can't be trusted at all unless you are already an expert in the thing you're asking it to "help" you with so you can correct the many mistakes it will make, but it's slower and, again, is **guaranteed **to make mistakes (hallucinations are built into what techbros are insisting on labeling as "AI", no matter how many resources you throw at it).
All of this at great environmental and human cost too.
I think his point is that this is less "news", and more "well, duh".
And even worse, it doesn't realise it and can't fix the errors.
Oh, so my sceptical, uneducated guesses about AI are mostly spot on.
As a computer science experiment, making a program that can beat the Turing test is a monumental step in progress.
However as a productive tool it is useless in practically everything it is implemented on. It is incapable of performing the very basic "Sanity check" that is important in programming.
The Turing test says more about the side administering the test than the side trying to pass it
Just because something can mimic text sufficiently enough to trick someone else doesn't mean it is capable of anything more than that
We can argue about it's nuances. same with the Chinese room thought experiment.
However, we can't deny that it the Turing test, is no longer a thought exercise but a real test that can be passed under parameters most people would consider fair.
I thought a computer passing the Turing test would have more fanfare, about the morality if that problem, because the usual conclusion of that thought experiment was "if you cant tell the difference, is there one?", but now it has become "Shove it everywhere!!!".
Oh, I just realized that the whole ai bubble is just the whole "everything is a dildo if you are brave enough."
yhea, and "everything is a nail if all you got is a hammer".
there are some uses for that kind of AI, but very limiting. less robotic voice assisants, content moderation, data analysis, quantification of text. the closest thing to Generative use should be to improve auto complete and spell checking (maybe, I'm still not sure on those ones)
I was wondering how they could make autocomplete worse, and now I know.
In theory, I can imagine an LLM fine tuned on whatever you type. which might be slightly better then the current ones.
emphasis on the might.
The Turing Test has shown its weakness.
Time for a Turing 2.0?
If you spend a lifetime with a bot wife and were unable to tell that she was AI, is there a difference?
The Turing test becomes absolutely useless when the product is developed with the goal of beating the Turing test.
it was also meant as a philosophical test, but also, a practical one, because now. I have absolutely no way to know if you are a human or not.
But it did pass it, and it raised the bar. but they are still useless at any generative task
this is expected, isn't it? You shit fart code from your ass, doing it as fast as you can, and then whoever buys out the company has to rewrite it. or they fire everyone to increase the theoretical margins and sell it again immediately
Anyone blindly having AI write their code is an absolute moron.
Anyone with decent experience (5-10 years, maybe 10+?) can absolutely fucking skyrocket their output if they properly set up their environments and treat their agents as junior devs instead of competent programmers. You shouldn't trust generated code any more than you trust someone fresh out of college, but they produce code in seconds instead of weeks.
I have tripled my output while producing more secure code (based on my security audits), safer code (based on code coverage and security audits), and less error-prone code (based on production logs and our unchanged QA process).
Now, the ethical issues and environmental issues, I 100% can get behind. And I have no idea what companies are going to do in 10 years when they have to replace people like me and haven't been hiring or training replacements. But the productivity and quality debates are absolutely ridiculous, as long as a strong dev is behind the wheel and has been trained to use the tools.
Consider: the facts
People are very bad at judging their own productivity, and AI consistently makes devs feel like they are working faster, while in fact slowing them down.
I've experienced it myself - it feels fucking great to prompt a skeleton and have something brand new up and running in under an hour. The good chemicals come flooding in because I'm doing something new and interesting.
Then I need to take a scalpel to a hundred scattered lines to get CI to pass. Then I need to write tests that actually test functionality. Then I start extending things and realize the implementation is too rigid and I need to change the architecture.
It is as this point that I admit to myself that going in intentionally with a plan and building it myself the slow way would have saved all that pain and probably got the final product shipped sooner, even if the prototype was shipped later.
What about my comment made you believe I was using gut feelings to judge anything? My ticket completion rate, number of tickets, story points, and number of projects completed all point to massive productivity gains.
The end of your comment was
Which is a general statement and not dealing with your specific circumstance. If a tool works for you, by all means keep using it.
However, broadly across software that is not the case. So the "productivity and quality debates" are not ridiculous .. the data supports the sceptics.
Absolute nonsense. Do people talk shit about hammers because some people keep hitting their hands with them? Do people complain about how useless ladders are, as one of the single most dangerous items in any household?
I don't think we should be putting these tools in the hands of junior devs - as the studies show, it hinders their productivity and learning. But to generally claim that they are bad tools with no upsides is just as ridiculous as the strawman you set up.
It depends on the task. As an extreme example, I can get AI to create a complete application in a language I don’t know. There’s no way that’s not more productive than me first learning the language to a point where I can make apps in it. Just have to pick something simple enough for the AI.
Of course the opposite extreme also exists. I’ve found that when I demand something impossible, AI will often just try to implement it anyway. It can easily get into an endless cycle where it keeps optimistically declaring that it identified the issue and fixed it with a small change, over and over again. This includes cases where there’s a bug in the underlying OS or similar. You can waste a huge amount of time going down an entirely wrong path if you don’t realize that an idea doesn’t work.
In my real work neither of these really happen. So the actual impact is much less. A lot of my work is not coding in the first place. And I’ve been writing code since I was a little kid, for almost 40 years now. So even the fast scaffolding I can do with AI is not that exciting. I can do that pretty quickly without AI too. When AI coding tools appeared my bosses started asking if I was fast because I was using one. No, I’m fast because some people ask for a new demo every week. Causes the same problems later too.
But I also do think that we all still need to learn how to use AI properly. This applies to all tools, but I think it’s more difficult than with other tools. If I try to use a hammer on something other than a nail, it will not enthusiastically tell me it can do it with just one more small change. AI tools absolutely will though, and it’s easy to just let them try because it’s just a few seconds to see what they come up with. But that’s a trap that leads to those productivity wasting spirals. Especially if the result actually somehow still works at first, so we have to fix it half a year later instead of right away.
At my work there are some other things that I feel limit the productivity potential of AI tools. First of all we’re only allowed to use a very limited number of tools, some of them made in-house. Then we’re not really allowed to integrate them into our workflows other than the part where we write code. E.g. I could trivially write an mcp server that interacts with our (custom in-house) ci system and actually increases my productivity because I could save a small number of seconds very often if I could tell an AI to find builds for me for integration or QA work. But it’s not allowed. We’re all being pushed to use AI but the company makes it really difficult at the same time.
So when I play around with AI on my spare time I do actually feel like I’m getting a huge boost. Not just because I can use a claude model instead of the ones I can use at work, but also just basic things like e.g. being able to turn on AI in Xcode at all when working on software for Apple platforms. On my work Macbook I can’t turn on any Apple AI features at all so even tab completion is worse. Or in other words, those realities of working on serious projects at a serious company with serious security policies can also kill any potential productivity boost from AI. They basically expect us to be productive with only those features the non-developer CEO likes, who also doesn’t have to follow any of our development processes…
Jeez, you aint joking about that brain injury :( I whish you good luck with your life. I am not trying to be an AH, i truly do whish you the best.
AI has made being OE insanely easy.
AI is shit. Details at 6!
Damn, I always think news at 11:00, but I don't think I've ever really watched news at 11:00
Schrodinger's AI: It is both useless shit that can only generate "slop" while at the same time being so effective, it is the reason behind 50,000 layoffs/going to take everyone's jobs.
Those two things aren't being claimed by the same people.
There are people with functioning brains, who are aware that AI is shit at programming, and there are managers who have been sold a sales pitch and believe that they can replace half of their software engineers.
AI doesn't actually need to be effective to cost a bunch of jobs, it just needs to have good salespeople. Those jobs will come back when the businesses which decided to rely on AI discover the hole they've dug for themselves. That might not be quick though, because there's no rule saying that major businesses will have competent leaders with good foresight.
ChatGPT is great at generating a one line example use of a function. I would never trust its output any further than that.
So much this. People who say ai can’t write code are just using it wrong. You need to break things down to bite size problems and just let it autocomplete a few lines at a time. Increase your productivity like 200%. And don’t get me started about not having to search through a bunch of garbage google results to find the documentation I’m actually looking for.
It’s laughable to me that people haven’t figured this out.
Personally I only do the "not search through garbage google results" part (especially now that it's clogged up with AI articles that don't even answer the question)
ChatGPT is great for that, I never have to spend 15 minutes searching up what's the function called to do X thing.
I really recommend to set the answers to be as brief and terse as possible. The base settings of a sycophant that generates a full article for every question are super annoying when you're doing actual work.
Not 200 %. Maybe 5-10 %. You still have to read all of it to check for mistakes, which may sometimes take longer than if you would have just written it yourself (with a good autocomplete). The times it makes a mistake you have lost time by using it.
It's even worse when it just doesn't work. I cannot even describe how frustrating it is to wait for an auto complete that never comes. Erase the line, try again aaaand nothing. After a few tries you opt write the code manually instead, having wasted time just fiddling with buggy software.
Agree with this. I personally don't use any sort of autocomplete whatsoever. When I have a question for the AI, I ask it, then I type the code from what I learnt.
Don't make the mistake of delegating work. Make the AI teach you what it knows.
How? "Hey, ChatGPT, write the thirty-second line of this function?"
Well not quite - I use ChatGPT more like to brainstorm ideas and sometimes I’ll paste a whole file or two into the prompt and ask what’s wrong and tell it the issue I’m seeing, it usually gives me the correct answer right away or after clarifying once or twice.
I use copilot for tab completion. Sometimes it finishes a line or two sometimes more. Usually it’s good code if it’s able to read your existing codebase as a reference. bonus points for using an MCP.
Warp terminal for intensive workflows. It’s integrated into your machine and can do whatever like implementing CICD scripts, executing commands, ssh into remote servers set up your infrastructure etc… I’ll use this when I really need the ai to understand my code base as a whole before providing any code or executing commands.
I don't know about ChatGPT, but Github Copilot can act like an autocomplete. Or you can think of it as a fancier Intellisense. You still have to watch its output as it can make mistakes or hallucinate library function calls and things like that, but it can also be quite good at anticipating what I was going to write and saves me some keystrokes. I've also found I can prompt it in a way by writing a comment and it'll follow up with attempt to fill in code based upon that comment. I've certainly found it to be a net time saver.
I'd never ask a friggin machine to do coding for me, that's MY blast.
That said, I've had good luck asking GPT specific questions about multiple obscure features of Javascript, and of various browsers. It'll often feed me a sample script using a feature it explains ... a lot more helpful than many of the wordy websites like MDN ... saving me shit-tons of time that I'd spend bouncing around a half-dozen 'help' pages.
I've been using it to code a microservice as PoC for semantic search. As I've basically never coded Python (mainly PHP, but can do many langs) I've had to rely on AI (Kimi K2, or agentic Claude I think 4.5 or 4, can't remember) because I don't know the syntax, features, best practices, and tools to use for formatting, static analysis, and type checks.
Mind you, I've basically never coded in Python besides some shit in uni, which was 5-10 years ago. AI was a big help - albeit it didn't spit out fully working code, I have enough knowledge in this field to fix the issues. As I learn mainly by practice and not theory, AI is great because - same as many YouTubers and free tutorials - it spits out unoptimized and broken code.
I am usually not using it for my main line of work (PHP) besides some boiler plate (take this class, make a test, make it look the same as this other test = 300 lines I don't have to write myself).
Jesus dude, this is what you're doing on Christmas eve? 😂
Don't bother. It's either bot or a bored troll.
I took a look at their comments and It's that alias_qr_rainmaker guy again!
I'm gonna tuck into my chocolate orange
Oh fuck I'm a bot you got me beeeep boooop list me up
Ai is literally just copy pasting. Like if you think about AI as a control C control V machine, it makes sense. You wouldn't trust a single fucking junior Dev that didn't actually know how to code because they just Ctrl C control V from stack overflow for literally every single line of code. That's all fucking AI is
And then it takes human coders way longer to figure out what’s wrong to fix than it would if they just wrote it themselves.
I find if I ask it about procedures that have any vague steps AI will stumble on it and sometimes put me into loops where it tells me to do A, A fails, so do B, B fails, so it tells me to do A...
I tend to get decent results by saying I want neither A or B when asking for C.
If AI is far better than humans, can you do yourself a favour, go talk to your little robot friends and leave us humans alone?
AAAAAAH I'm being gaslit oh nooooo
Source:
I made it the fuck upanecdoteCan I be on the list too? Please?
You're very antagonistic in your approach to other people. Which results in very low scoring comments. And you seem to use your brain injury as a shield to not better yourself. Oh wait, a bot would use an em—dash here and there. — there's two now.
On top of all the damage AI unleashed into the world, there's that: it ruined em dash for people who appreciate typography.
I didn't even know what an em dash was before ChatGPT, so I guess that's a positive result of LLM.
If you liked em dashes, you'll love ligatures.
I'm not a programmer, but I've dabbled with Blender for 3D modeling, and it uses Node trees for a lot of different things, which is pretty much a programming GUI. I googled how to make a shader, and the AI gave me instructions. About half of it was complete nonsense, but I did make my shader.
You need to babysit and double check everything it does. You can’t just let it loose and trust everything it does.
As expected
Not my code though. It contains a shit ton of bugs. When I am able to write some of course.
Nah, AI code gen bugs are weird. As a person used to doing human review even from wildly incompetent people, AI messes up things that my mind never even thought needed to be double checked.
The things I have seen from devs who thought they could lie and pretend they didn't use AI...
Human bugs >>> AI bug slop
Human bugs are more beautiful
All (human) Bugs Are Beautiful
I'll go ahead and file this under "duh".
No shit, Sherlock (c)
This is news?
Microsoft: Let's have it rebuild our most well known product from the ground up!
But you see. That’s the solution. Now you pay foreigners to clean up the generated code by offshoring the engineers. At 1/100 the cost.
Hey don't worry, just get a faster CPU with even more cores and maybe a terabyte or three of RAM to hold all the new layers of abstraction and cruft to fix all that!
Yeah. No shit. I used an LLM's "help" to make fin.
It got me reading and debugging more than 10 times the [bad] code, per day, than I had in the entire prior 10 years of using fish. [And reading the documentation way more too, learning a lot.]
Oh but it's so effortless. HA! Debugging takes a lot more effort. And then still have to just re-write it all yourself any way.
Still, it's a good learning experience.
Dear AI,
Thanks for being so shit.
Taught me a lot.
So this article is basically a puff piece for Code Rabbit, a company that sells AI code review tooling/services. They studied 470 merge/pull requests, 320 AI and 150 human control. They don't specify what projects, which model, or when, at least without signing up to get their full "white paper". For all that's said this could be GPT 4 from 2024.
I'm a professional developer, and currently by volume I'm confident latest models, Claude 4.5 Opus, GPT 5.2, Gemini 3 Pro, are able to write better, cleaner code than me. They still need high level and architectural guidance, and sometimes overt intervention, but on average they can do it better, faster, and cheaper than me.
A lot of articles and forums posts like this feel like cope. I'm not happy about it, but pretending it's not happening isn't gonna keep me employed.
Source of the article: https://www.coderabbit.ai/blog/state-of-ai-vs-human-code-generation-report
I am a professional software engineer, and my experience is the complete opposite. It does it faster and cheaper, yes, but also noticeably worse, and having to proofread the output, fix and refactor ends up taking more time than I would have taken writing it myself.
A later commenter mentioned an AI version of TDD, and I lean heavy into that. I structure the process so it's explicit what observable outcomes need to work before it returns, and it needs to actually test to validate they work. Cause otherwise yeah I've had them fail so hard they report total success when the program can't even compile.
The setup I use that's helped a lot of shortcomings is thorough design, development, and technical docs, Claude Code with Claude 4.5 Sonnet them Opus, with search and other web tools. Brownfield designs and off the shelf components help a lot, keeping in mind quality is dependent on tasks being in distribution.
In web development it's impossible to remember all functions, parameters, syntax and quirks for PHP, HTML, JavaScript, jQuery, vue.js, CSS and whatever else code exists in this legacy project. AI really helps when you can divide your tasks into smaller steps and functions and describe exactly what you need, and have a rough idea how the resulting code should work. If something looks funky I can ask to explain or use some other way to do the same thing.
That sounds almost like an AI version of TDD.
And now instead of understanding the functions, parameters, syntax and quirks yourself, to be able to produce quality code, which is the job of a software engineer, you ask an LLM to spit out code that seem to be working, do that again, and again, and again, and call it a day.
And then I'll be hired to fix it.
I have also used the latest models and found that I've had to make extensive changes to clean up the mess it produces, even when it functions correctly it's often inefficient, poorly laid out, and is inconsistent and sloppy in style. Am I just bad at prompting it or is your code just that terrible?
The vast majority of my experience was Claude Code with Sonnet 4.5 now Opus 4.5. I usually have detailed design documents going in, have it follow TDD, and use very brownfield designs and/or off the shelf components. Some of em I call glue apps since they mostly connect very well covered patterns. Giving them access to search engines, webpage to markdown, in general the ability to do everything within their docker sandbox is also critical, especially with newer libraries.
So on further reflection, I've tuned the process to avoid what they're bad at and lean into what they're good at.
Well, yeah. It also took 100x the time to write.
Vibe coding is only really useful for a coder. Because you can understand and correct it.
what's funny is that this was predicted to be that way even before AI-generated code became an option. Hell, I remember doing an assessment back in early 2023 and literally every domain expert i talked with said this thing - it has its use, but purely supplemental and you won't use it on some fundamental because the clean-up will take more time than was preserved. Counterproductive is the word.
Shocker.
I couldn't program a single line of code if my life depended on it, and I could have told you that.
Not if Yandev has anything to say about it.
Depends on who the human is haha
Yeah, you've clearly never seen my code!
AI code is great for getting over a hump, something you're stuck on. Used ChatGPT (not the best for coding, I know) to help on a PowerShell script. There was exactly two references on the internet for what I wanted to do (Google Calendar/Sheets integration). Spent hours on the problem.
ChatGPT gave me two things: One solution I didn't know was a thing, another was a twist I hadn't thought of. For giggles, I plugged the whole script in. Guess what? Failed instantly. Because of course it did.
No. LLMs don't write working code. Yes. They can help you, assuming you know what you're doing in the first place. But here's the crux of using AI:
It does not, and cannot, give a shit about edge cases, user error and security.
I wrote a simple PS script to swap my TV screens around for work, play and movies. Rolled it out in 30 minutes. Took me 2 more hours to stupid proof it, test it, wrap it an exe, make an icon, deploy it, all that. AI can't do any of that.
what would socialists/communists do?
I really, really, want to stop seeing posts about:
I swear these are the only things that the entire Lemmy world wants to talk about.
Maybe I should just go back to Reddit... Fuck Spez, but at least there is some variety.
your frontend of choice probably has some option to hide posts containing specific keywords
Yeah, good point. Was hoping to avoid downloading another random app, but at this stage, I guess It's something I should look into.
Yes, please just hide these. We ignoring these issues at large is how we got to where we're at and it'll continue getting worse if we just stop talking about it. But you need to do what you can to take of yourself, first and foremost.
Ignoring?
I've read at least 7 iterations of this exact same article. There is absolutely no originality here at all.
I'm not talking about this particular article, I'm talking about the sentiment of wanting to shut off the conversation about those topics you mentioned. Like I said, do what you have to do and take care of your own mental health. I assume that if you were only talking about reposting of this article you'd not have mentioned your whole list of topics.
Microsoft could write an AI agent to filter threads based on context you don't like. Come to think of it, Megagenius Elon Musk already has one he wrote to censor anti-Israel posts on Trump's Truth Social. There, I think I got them all.... Happy holidays!
But as long as the kids get to keep calling themselves “artists” and “musicians”, its all good.
what would marx do?
Do not ask a corpse for advice, the question is what are we going to do?
Boycott is a good first step, although I am not sure if it is better to boycott them or use their free tier to have the most deranged BS conversation that will consume their resources, eat at their scare cash reserves and when they use it in training, it will poison their data.
AI doesn't generate its own code, humans using AI generate code. If a person uses AI to generate code and doesn't know good practices then of course the code is going to be worse.
People expect perfection right out of the gate.
I mean damn, AI has only been able to write something resembling code for a few years now. The fact that this is even a headline is pretty amazing when you think about it.
A lot of LLM hype is wrapped up in how well it can write code. This hype is being used by corporations to justify pouring mind boggling amounts of money into the tech in the hopes that they can lay off all their staff.
I reserve the right to hate this state of affairs and enjoy seeing every headline that shows just how much of a pipe dream it is.
I don't mind imperfections while they work out the kinks. I dislike dismantling industries in favor of something that doesn't work yet.
AI apologia is nearly as cringy as MAGA apologia. Stop doing this.
Yes bitching about it is going to make the fucking billionaires quit using it.
You prefer only they get it? Because that’s what’s going to happen if you somehow got any part of your wish.
This isn’t making the point you think it’s making.
Haven't you ever seen Star Trek and been amazed by the holodeck? How do you think they got there?
WTF is up with people today using fictional bullshit as grounds to argue dumb shit?
Star Trek isn’t real. Please tell me you understand this. It’s very important to you that you do.
No need to be deliberately obtuse and insulting.
It's pretty clear that the holodeck's ability to create a complicated program from a basic prompt is analogous to LLM output.
If too many people have your attitude, we'll never get there.
Oh for fucks sake. You’re a troll and I fell for it. That’s on me.
I'm having a sensible conversation and you're freaking the fuck out for no reason.
Freaking out? I’m laughing my ass off! Either you’re a damn good troll, or you actually think Star Trek is real look at a real future.
Either way, it’s funny!
Magnets clearly.