Microsoft says Copilot will 'finish your code before you finish your coffee' adding fuel to the Windows 11 AI controversy that's still raging
https://www.techradar.com/computing/windows/microsoft-says-copilot-will-finish-your-code-before-you-finish-your-coffee-provoking-another-barrage-of-anti-ai-and-windows-11-sentimentOpen linkView original on reddthat.com438
Comments110
It says it will finish the code, it doesn't say the code will work.
Also just because the code works, doesn't mean it's good code.
I've had to review code the other day which was clearly created by an LLM. Two classes needed to talk to each other in a bit of a complex way. So I would expect one class to create some kind of request data object, submit it to the other class, which then returns some kind of response data object.
What the LLM actually did was pretty shocking, it used reflection to get access from one class to the private properties with the data required inside the other class. It then just straight up stole the data and did the work itself (wrongly as well I might add). I just about fell of my chair when I saw this.
So I asked the dev, he said he didn't fully understand what the LLM did, he wasn't familiar with reflection. But since it seemed to work in the few tests he did and the unit tests the LLM generated passed, he thought it would be fine.
Also the unit tests were wrong, I explained to the dev that usually with humans it's a bad idea to have the person who wrote the code also (exclusively) write the unit tests. Whenever possible have somebody else write the unit tests, so they don't have the same assumptions and blind spots. With LLMs this is doubly true, it will just straight up lie in the unit tests. If they aren't complete nonsense to begin with.
I swear to the gods, LLMs don't save time or money, they just give the illusion they do. Some task of a few hours will take 20 min and everyone claps. But then another task takes twice as long and we just don't look at that. And the quality suffers a lot, without anyone really noticing.
Great description of a problem I noticed with most LLM generated code of any decent complexity. It will look fantastic at first but you will be truly up shit creek by the time you realise it didn't generate a paddle.
Big baffling facepalm moment.
If they would at least prefix the changeset description with that it'd be easier to interpret and assess.
Who hasn't encountered that one jerk who builds only new code to impress management, and never maintains or fixes existing code? I think of them as proof-of-concept posers. They make things that look flashy, impress the execs, and barely work for a single use care, then dump all the bugs, maintenance and actual architecture on the other devs. LLMs are going to be a gift to these people and a pain for everyone who actually knows how to engineer things well. They'll encourage this kind of shallow flashiness and make the maintenance problems worse, but the execs will be convinced that only the LLM posers are productive and everyone else is sitting idle.
They’ve been great for me at optimizing bite sized annoying tasks. They’re really bad at doing anything beyond that. Like astronomically bad.
Why would unit tests not be written by the same person? That doesn’t make a lot of sense…
They did say why they're doing it
Did that not make sense to you?
I usually wouldn't do that, because it's a bigger investment. But it certainly makes logical sense to me and is something teams can weigh and decide on.
I was going to say. The code won’t compile but it will be “finished “
A couple agent iterations will compile. Definitely won't do what you wanted though, and if it does it will be the dumbest way possible.
Yeah you can definitely bully AI into giving you some thing that will run if you yell at it long enough. I don’t have that kind of patience
Edit: typically I see it just silently dump errors to /dev/null if you complain about it not working lol
And people say that AI isn't humanlike. That's peak human behavior right there, having to bother someone out of procrastination mode.
The edit makes it even better, swiping things under the rug? Hell yeah!
Ooh, unemployment! How exciting! I love Microsoft now.
Seriously who the hell are they trying to sell this to?
Are they just that desperate to keep the hype train going?
Business owners. People that don't want to spend money on annoying stuff like wages.
CEOs are convinced that if they can get rid of those pesky expensive engineers that idea people will magically make things work.
AI is the promise of slaves 2.0
The circlejerk of tech bros and busidiots who haven't built a damn thing in their lives.
retail suckers and losers who buy into the AI crap to keep the "bubble from bursting overnight"
There are entire media agencies that only do vibe coding. And that might be enough for a one-off event but many of them „develop“ long term solutions without knowing the code or it‘s vulnerabilities so it‘s safe to say their existence will be short lived.
Copilot, turn on the gas stove without the pilot. Copilot, in 3 hours light the pilot.
My Windows automatically read these instructions from my screen and I died!
You forgot to follow it up with “copilot: open windows” then.
Username checks out
Copilot keeps finishing my code for me in near real time... it completely disrupts my train of thought and my productivity dropped tremendously. I finally disabled it.
I LIKE writing code, stop trying to take the stuff away that I WANT to do and instead take away the stuff I HATE doing.
What I don't want AI to do:
What I want it to do:
They do have ones that will review your prs. That's pretty neat
Yeah, I just wrote a blog post comment about how I enjoy using Copilot. But that's when I explicitly ask it a question or give it a task. The auto complete is wrong more often than it's right.
Probably doesn't help that if it was tedious, boilerplate code I would have already explicitly asked it.
I like auto complete because I'm a terrible speller I'd write "int countOfCommplixThang", but auto complete guesses "int countComplexThing" Sometimes it even comes up with a better name than I would
That old autocomplete is great. It's specifically the AI autocomplete that's less useful.
The AI is doing it though. I type "in [tab]" and I get the whole line. Sometimes I don't even have to type anything.
I've never been able to get an AI promt to write useful code though.
I wish I could get it to stop finishing comments for me. It’s like some jackass is trying to complete my sentence for me but gets it completely wrong every time and it breaks my train of thought.
What they forget to mention is that you then spend the rest of the week to fix the bugs it introduced and to explain why your code deleted the production database...
yes but all the code will be wrong and you will spend your entire day chasing stupid mistakes and hallucinations in the code. I'd rather just write the code myself thanks.
Yeah! I can make my own stupid mistakes and hallucinations, thank you very much!
I had a bit of a breakthrough with some personal growth with my code today.
I learned a bit more about entity framework that my company is using for a project, and was able to create a database table, query it, add/delete/update, normal CRUD stuff.
I normally work mostly on front end code, so it was rewarding to learn a new skill and see the data all the way from the database to the UI and back - all my code. I felt great after doing a code review this afternoon to make sure I wasn’t missing anything, and we talked about some refactoring to make it better.
AI will never give you that.
No, but it can help a capable developer to have more of those moments, as one can use LLMs and coding agents to (a) help explain the relationships in a complicated codebase succinctly and (b) help to quickly figure out why one's code doesn't work as expected (from simple bugs to calling out one's own fundamental misunderstandings), giving one more time to focus on what matters to oneself.
Wow, well it's absolutely terrible at A. B is worth a shot, but it's 50/50 to bullshit you in my experience.
Yeah - there's definitely a GIGO factor. Throwing it at a undocumented codebase with poor and inconsistent function & variable names isn't likely to yield great revelations. But it can probably still tell you why changing input X didn't result in a change to output Y (with 50k lines of code in-between), saving you a bunch of debugging time.
TBH, it's not really that great at that. Is average at best and grossly misleading and flat out wrong at worst. It may bring slight speedups for average development on boring legacy enterprise code, but anything really novel and interesting? Detrimental.
Most code on the planet is boring legacy code, though. Novel and interesting is typically a small fraction of a codebase, and it will often be more in the design than the code itself. Anything that can help us make boring code more digestible is welcome. Plenty of other pitfalls along the way though.
Fair enough, that's true. I guess my gripe is with the narrow use case and the debugging and/or prompt/context tuning to get what you want. I still feel that if you don't get what you want on the first try, it's faster to write it yourself than spending time "debugging" the input and maybe get a 60% chance on correct output, which in most cases, still needs debugging. And god forbid, a framework is rewritten.
I just wished it was a bit better before we hit the plateau of diminishing returns.
Hah, yeah. Vibe coding and prompt engineering seem like a huge fad right now, although I don't think it's going to die out, just the hype.
The most successful vibe projects in the next few years are likely to be the least innovative technically, following well trodden paths (and generating lots of throwaway code).
I suppose we'll see more and more curated collections of AI-friendly design documents and best-practice code samples to enable vibe coding for varied use-cases, and this will be the perceived value add for various tools in the short term. The spec driven development trend seems to have value, adding semantic layers for humans and AI alike.
AI can help you be more agile in getting out a PoC but vibe coding always ends up eating itself and you either aren’t capable enough to fix it (because you are a vibe coder) or you spend more time on the back 9, trying to clean up the code so you don’t have so many hacks and redundancy because the AI was too literal or hallucinated fake libraries that return null or its context window expired and it wrote 5 different versions of the same function
Eh, I've enjoyed writing a SQL query and having AI translate it to Linq. I've had at least one work directly, very clear on what it's doing, just with Linq's odd syntax. The other query was more complicated and wasn't something that translated well to Linq. I may have had to split that into two Linq queries.
Then again, I wouldn't count translating psuedocode (or SQL) as really vibe coding. To me "vibe coding" means you're not really looking at the code it produces.
It's a changing world, and there is going to be an ever increasing amount of AI slop out there, and even more potential programmers who won't make the leap due to the crutch.
At the same time, there are always people who want to and will learn in spite of the available crutches the latest tech revolution brings.
There will also be many good engineers who will exploit the tech for all its worth while applying appropriate rigour, increasing their real productivity and value manyfold.
And there will be many non-programmers who can achieve much more in their respective fields, because AI tools can bridge gaps for them.
Hopefully we won't irreversibly destroy ourselves and our planet while we're at it. 🙈
It will offer an explanation, one that sounds consistent, but it's a crap shoot as to whether or not it accurately described the code, and no easy of knowing of the description is good or bad without reviewing it for yourself.
I do try to use the code review feature, though that can declare bug based on bad assumptions often as well. It's been wrong more times than it caught something for me.
Hm? Oh, I obviously misread the room. It seems I interrupted a circle jerk? My apologies.
I thought you made a good point. I have decades of experience and I find LLMs useful for the things you described.
I have a suspicion that the guy took issue with my use of "one" instead of "you", more-so than the content. Maybe it came across as uppity.
I was finished with Windows before Microshit finished Copilot.
"Microsoft Says Copilot" is like a Two Sentence horror story for me now.
Love how they're pretending that an LLM is useful for any task that needs precision.
Actually it won't be finishing anything because code is disposable now and nobody cares what trivial app somebody can churn out
Because you won't have time to drink that coffee if you put this code into production
Writing code is the reward for doing the thinking. If the LLM does it then software engineering is no fun.
It's like painting - once you've finally finished the prep, which is 90% of the effort, actually getting to paint is the reward
What a great way to frame it, I love this! I typically spend something like 60-80% of time available for a given task thinking through approaches and trade-offs, etc. Usually there comes a point when the way forward becomes clear, even obvious.
After that? Bliss. I'm snapping together a LEGO set I designed, composed of pieces I picked (maybe made one or two new ones!), and luxuriating in how it all feels, when put together.
And it will leave you debugging strange code for two weeks afterward.
Oh wow I didn't know about that butt plug thing. I'm playing in a chess tournament soon so that could come in handy
So why don't they use it to unfuck Windows 11... before I finish my coffee?
It was the AI that messed it up to begin with lol. Vibe coding has often required coders having to go back and spend even more time fixing it then if they just did it themselves.
best they can do is put more AI
I already finished my coffee too. :-/ Though I suppose I could throw on another pot while we wait.
I've finished several coffees since you posted this.... pretty sure win11 is still fucked
If you want it done before you finish your coffee, better tell it to start from scratch
Just enjoying this popcorn.
But, will it work, huh? HUH?
I can also type a bunch of random sentences of words. Doesn’t make it more understandable.
but can YOU do it before I finish my coffee?
Some models are getting so good they can patch user reported software defects following test driven development with minimal or no changes required in review. Specifically Claude Sonnet and Gemini
So the claims are at least legit in some cases
Oh good. They can show us how it's done by patching open-source projects for example. Right? That way we will see that they are not full of shit.
Where are the patches? They have trained on millions of open-source projects after all. It should be easy. Show us.
That's an interesting point, and leads to a reasonable argument that if an AI is trained on a given open source codebase, developers should have free access to use that AI to improve said codebase. I wonder whether future license models might include such clauses.
Free access to the model is one thing but free access to compute for it is another
Are you going to spend your tokens on open source projects? Show us how generous you are.
I'm not the one trying to prove anything, and I think it's all bullshit. I'm waiting for your proof though. Even with a free open-source black box.
What's a free open source black box?
Some kind of bad joke that went way over your head. Where are your merge requests?
At work, the software is not open source.
I would use it for contributions to open source projects but I do not pay for any AI subscriptions, and I can't use my employee account for copilot enterprise for non-work projects.
Every week for the last year or so I have been testing various copilot models against customer reported software defects and it's seriously at a point now where with a single prompt Gemini pro 2.5 is solving the entire defect with unit tests. Some need no changes in review and are good to go.
As an open source maintainer of a large project I have noticed a huge uptick in PRs which has created a larger review workload, I'm almost certain these are due to LLMs. Quality of a typical PR has not decreased since LLMs have become available and I am thus far very glad
If I were to speculate I'd guess the huge increase in context windows has made the tools viable, models like GPT5 are garbage on any sizable code bases
Will it execute . . . probably.
Will it execute what you want it to . . . probably not.
You really have no idea, I'm working with these tools in industry and with test driven development and human review these are .performing better than human developers
YMMV.
I mean it gets there in the end but it's often three of four prompts before it provides working code for a relatively simple powershell script. Can't imagine that it scales to complex code that well at the moment, but then again I'm not a coder.
You’re pretty much spot on
These fuckers at MicroShit have lost all the ability needed to read a room.
When do you reckon they could last do that?
Maybe after windows 8? Last time I can remember.
I'd say 97 was the last time.
My problem is that the dev and stage environments are giving me 502 gateway errors when hitting only certain api endpoints from the app gateway. My real problem is devops aren't answering my support tickets and telling me which terraform var file I gotta muck with and tell me what to fix on it. I'm sure you'll be fixed soon though right copilot?
By the headline statement, that it should be complete and works 100%. Big doubt.
No, just complete. Whatever the dude does may have nothing to do with what you needed it to do, but it will be "done"
Depends. If it's a script that will like, cut your video file every 10 seconds with ffmpeg or something simple. Yeah it will one-shot it.
But... But I don't want it to. 😮💨
Does its ai learn from people using vscode?
Yes
Big over-promise. We're heavily incentived to use an AI coding agent at work. I try to be optimistic and treat it like a tool to help me do things I already know how to do but a little bit faster. It takes multiple iterations of "no, this still isn't working" to get something that I can touch up and push for review. The idea that I can prompt it and then step away for ten minutes to make coffee and return to an app is ludicrous.
Maybe one day that will be possible. Then I'll find a new job I guess
It's great at bullshitting that it did what you wanted, even if it obviously didn't, which I guess is what counts for results at Microsoft.
It would be much better if they treated it as the slightly better (yeah, I said it) auto complete that it is instead of the beginning of fucking sky net -- which was supposed to be a bad thing anyway, remember?
But that wouldn't move the needle on all of the share prices, so instead we have to pretend it can do people's jobs when it fucking obviously cannot.
So, instead they keep pushing this AI (auto-complete insanity), and keep burning more and more cash. Imagine if we just put a portion of these billions (approaching trillions) into anything that could actually help anyone. Or don't, because it's pretty fucking depressing to think about.
I would rather paint a portrait by myself, spending the time to do it, rather than asking some computer prompt to spit me out a picture. Same logic applies with coding for me.
If thats what they are aiming at, I feel like their AI is actually suppose to be the pilot and the user the copilot
I read “users respond with mercyless trolling” in the teaser, I have to open the article.
I can drink coffee pretty slow, but I don't think I can drink it that slow
Good thing finishing your coffee is many sips. Because Copilot certainly doesn't feel fast. It often feels so slow you wonder whether waiting is worth it.
Technically true, but nobody said the code will be at all functional. I'm pretty sure I can finish about 800000 coffees before Copilot generates anything usable that is longer than 3 lines.
To safely put it out just pee on it.
In my experience, which consists of using copilot for about ten minutes, literally every single suggestion is wrong, and if you're not careful it'll insert the shitty code and then you have to go back and find out why the code isn't working.
I'd rather have Lizzo shum on my face than use copilot
I do NOT want to have ditch Windows after this long. Microsoft. Please don't just carve a big, Copilot-pilot-shape hole out of Windows and weld it in there, expecting that that is somehow what your users want.
Of course, I do look forward to the brave new era of "Sam Altman will shut you down unless" being the new "run as administrator"
...but what am I going to do while it's compiling?
I would like to someday use AI to remaster Stars!, Magic Carpet, and Judgment Rites. However, it won't be through co-pilot, because I fundamentally don't trust Microsoft.
In any case, I think genuine "hands off" development from an AI would be at least a decade off. Partially just for it to have the ability, but also for local hardware to support it. (I only use local AI, but a 100b like GLM is slow as heck on my gaming rig.)
My boss literally every day: wow look at this new ai tool I just found!
Make it de the shit I don't want to do, then we'll talk
The only time I use copilot is when I am bored at work and I make jokes with it.
yeah but then you have to fix everything in the code that they didn't get right.
like using it to automate a shell is fine; but trusting it blindly and treating it as the finishing product? you're delusional.