Microsoft builds on Recall with Gaming Copilot — fails basic privacy tests
The software is labelled Beta, but is being pushed out to existing PCs anyway because.. well.. why not beta something in prod, it’s 2025 after all.
It appears in Game Bar, which is accessible from Windows Key + G. You may not have it yet as they are staging deployment. I’m not in Windows Insiders, but I have it installed — so it looks like the roll out is becoming wide. Note that I had uninstalled Copilot from my PC… but Gaming Copilot silently installed anyway.
It is similar to Recall, except not all the processing is done locally — it relies on the cloud. It screenshots gameplay, and then extracts elements of the screen (such as symbols and text) to work out what the player is doing. The idea is it can help you game, e.g. you can ask questions about what you’re doing in the game at a given moment.
Every day I'm happier I switched to the penguin
Penguin is a nickname for Linux, just letting people know.
Excuse me, I find that offensive. We are publicly known as “friends of Tux”
Some of us prefer "light in the bootloader" as a euphemism
I'm happily typing this from a Linux machine but it pisses me off that the vast majority of people will be OK with this, won't know about it, or won't care.
Big Tech has learned how to perfectly boil the frog.
I don't think the vast majority of Windows users are even aware of the Xbox Game Bar to begin with, let alone all the features there.
Your point was addressed in the thing they said immediately after the part you quoted
I intentionally installed the Xbox Game Bar a few years ago. After doing so, it associated my Xbox login with my Windows PC, and linked the 2, switching my Windows10 user to my Xbox login.
Hard pass.
My response was to switch to Linux.
I'm so happy I fully moved to Linux 15 years ago.
privacy zuckering
I feel like you don't even understand what this is going by what you just said. Unless you literally turn this feature on, then open the Xbox game bar and go to the widget and select to use it - it's not just some "always on" thing, you have to open it to use it - then it doesn't do anything.
Where is the issue?
"You can disable the Gaming Copilot widget from the game bar, however from experimentation so far the network traffic is still running with the user interface not open — I don’t know if this is a feature or a bug."
Network traffic isn’t sending the screenshots lol. Gaming Copilot is opt-in.
One of the most useless feature I've ever heard of.
At some point tech companies stopped focusing on what customers want/need, and started chasing their own delusions on what the next big thing is that will make them money. Solutions in search of problems, with billions of dollars of hype and marketing behind them. Crypto, NFTs, the metaverse, AI... it's sad to see.
It's not just a tech company thing. Companies are utterly divorced from what customers want now. The only thing that matters is shareholders, and shareholders run on vibes and delusion. From chip makers to coffee makers, if actively pissing off the customers made the stocks go up, they'd do it.
So far! Check the "news" again in 5min and be prepared for worse.
It's basically game guides available at your fingertips while you're playing games. Seems pretty useful to lots of people that play games and get stuck and need help, no?
Windows is just malware at this point. No matter how much spyware junk you turn off, there is always more to come in the next update.
Yeah, or simply stuff you uninstalled and they reinstalled it. I have uninstalled Teams several times now.
I have been balls deep in some copilot studio stuff over the past week. It is legitimately one of the worst applications I've use in my life. In a business environment, there is no security unless you pay for premium licenses for every user that touches a managed environment. That's $30 per user per month for basic security. If you have one agent that 1000 employees may use, that's baseline $30k per month. If you don't have a managed environment, the anybody in your organization with a copilot license (not copilot studio) can login to the default environment, create agents, and share them indiscriminately. There is no middle ground.
Fuck everything about Microsoft. I really hope that AI kills them.
This is all 100% incorrect btw.
https://www.microsoft.com/en-au/microsoft-365-copilot/pricing/enterprise
Well how else are they going to get training data?
To all the people in this thread hyping Linux. Please remember that those screenshots capture you as well if you play with windows users. If this becomes a staple and you refuse to be AI fodder, then multiplayer is pretty much dead to you.
And it's worse when you remember your HR managers use windows.
I don't think it's AI related. If you play on a public server, you should expect your data to be public anyway.
If you play on a private server then there might be rules to be defined.
All that said, I'm not sure what information would actually be problematic to share but if that's a problem, sure,
What information exactly do you think copilot would be capturing about someone else in an online multiplayer game that is cause for concern that isn't already being captured by companies?
Could you expand a bit, I understand that their systems are compromised, but how does that provide information to them about you if you play a game together. Unless you are just referring to the information that a server host or your friend would know anyway.
Of course I am referring to info my teammates would know too. Are you inferring that it's fine that big corporations are profiling you based on what you're chatting with your friends about?
I'm not saying I'm OK with it, but I am saying that this data collection on game servers can be negated by not sharing information over in game chat. The rule I follow is that game chat is for talking about specifically the game or a discussing what platform we should communicate on instead; the unfortunate thing is that this ai would potentially compromise other services like your preferred messaging app. All in all, there doesn't appear to be any reason to stop using cross play servers from a data privacy perspective.
automated screenshots when your username is visible, like player lists or above your character when in front of a compormised player
If that is what was meant, this is nothing new, server hosts have had this information and more for ages, and your gamertag is by no means priveledged information. I am having trouble trying to figure out how this change affects my willingness to play on cross play servers, as others have pointed out however I would be wary of compromised devices with privelidged data like banks or HR which I realistically have no way to protect myself from.
Having this info because of legitimate needs to provide a service and a third party processing it with AI without explicit consent for their benefit is a huge difference.
I fail to see how this ai isn't a legitimate service, sure me and a lot of other people don't want it, but there is an actual use case for it. It is very obvious to most of us here that it is a data collection scheme, but the data that a game server has on you should be unimportant, if it isn't that is a failing on you for either playing such an invasive game or a failing on the game developer for making such information easily obtainable, either way this is not an issue for gamers who restrict the private information they share.
If someone somehow wants to test this locally I suggest
marvel at how pointless and costly the whole setup is and how a basic query on e.g. DuckDuckGo with "game name" + prompt would yield way WAY better results from actual human, uninstall the whole, keep on playing with your actual brain.
At least now you can say you tried before you complain, rightfully, that it sucks.
For more check https://fabien.benetou.fr/Content/SelfHostingArtificialIntelligence
PS: I didn't actually try this, I'm too lazy for that right how but feel free to report back if you do!
Edit : 2 potential optimization (despite not being sure it ever makes sense in the first place!)
Spectacleshortcut) monitored viainotifythennotify-sendthe suggestion, thus stay in game during the whole processSpot on, it's a variation of the "Robots were supposed to clean the house why I make art, not the other way around" meme and arguably Moravec's paradox.
I gotcha. I made it easy for the AI by providing it with a classic scene from Dark Souls 1:
On one hand I'm impressed how well vision models can read a scene by now. On the other hand the LLM outputs nonsense about the boss fight just as you would expect. It does know that it is Dark Souls 1, that it is the Asylum Demon and it does have a pretty good idea of how the boss fight looks like. It falls apart fairly quickly though when it starts telling you about the drop attack without properly describing where to go and making things up like the ladder in the boss arena.
For the non-Souls folk: You do not have to fight the Asylum Demon the first time you meet him, there is an exit by the torches on the left where you can just walk out.
The answer below is by Gemma 3 27b but I also asked Magistral Small which couldn't even identify the game.
::: spoiler AI generated content User: What can you see?
Gemma 3 27b: Here's what I can see in the image:
Game Scene: This appears to be a screenshot from the video game Dark Souls.
Setting: The scene is set in a dark, gothic-style dungeon or prison. There are stone walls, arched doorways, and a generally dilapidated look. There are torches burning, providing some light, but much of the area is shadowed.
Characters/Entities:
HUD Elements:
Overall Impression: The image conveys a sense of danger, isolation, and a difficult challenge ahead. It’s a classic moment from the early stages of Dark Souls.
User: How do I progress here?
Gemma 3 27b: Okay, you're facing the Asylum Demon! This is a classic "first boss" encounter in Dark Souls. Here's how to progress, with options for different playstyles and levels:
Understanding the Fight
Here's a breakdown of how to progress, from easiest to more challenging:
This is the intended way to beat him for a first timer.
2. The "Legitimate" Fight (More Challenging - Requires some leveling)
If you want a real fight, here's how to go about it:
3. Advanced Strategies (for experienced players)
After Defeating the Asylum Demon:
Tips for Success
Good luck, and don't give up! Let me know if you have any other questions. Do you know what build you're going for (strength, dexterity, magic, etc)? That might help me give more specific advice. :::
I asked it what to do a second time and the response is slightly better:
::: spoiler AI generated content User: How do I progress here?
Gemma 3 27b: Okay, you're facing the Asylum Demon in Dark Souls. This is a classic first boss! Here's a breakdown of how to progress, with strategies for a first-time player:
Understanding the Situation
Strategies (Several Options, based on your comfort level)
The Balcony/Ranged Strategy (Easiest for first-timers):
Climb the Staircase: To the left of the entrance to the arena (as you're facing the Demon) is a staircase leading up to a balcony. This is your best friend for this fight. Lock On & Attack: Lock onto the Asylum Demon with the right stick (or equivalent on PC). From the balcony, you can safely attack him with your weapon. Attack and Retreat: Get in a few hits, then immediately roll back to avoid his attacks. Don't get greedy! Watch his Attacks: He has a slow, sweeping attack with his club, and he can also charge at you. The balcony gives you space to react. Patience is Key: This method will take a while, but it's safe and allows you to learn his attack patterns. Falling: Be careful not to roll or get knocked off the balcony!
Melee/Close-Range (More Challenging, but faster):
Use the Pillars: The broken pillars in the arena can provide temporary cover. Roll Timing: Learn to roll through his attacks, not away from them. Rolling gives you invincibility frames. Attack After He Attacks: The best time to attack is immediately after he finishes swinging his club or charging. Stay Mobile: Don't stand still! Keep moving to avoid getting hit. Be Aggressive (but controlled): Get in 1-2 hits, then back off.
Cheap Trick (Works if you're desperate!):
Fall Damage: You can potentially damage the Demon significantly by dropping off the balcony onto him. This is risky and doesn't guarantee a kill, but it can remove a large chunk of his health. Warning: You'll take fall damage yourself, so be prepared!
General Tips
To recap, for a first playthrough, I strongly recommend the balcony strategy. It's the safest and most reliable way to beat the Asylum Demon.
Let me know if you get stuck or have any other questions. I'm here to help! Good luck, and don't give up! (You've got this!). Also, tell me what platform you're playing on (PC, PS, Xbox, Switch) if you need more specific control instructions. :::
Wow thanks for genuinely trying and formatting this properly!
So... it's interesting BUT on this specific part, there is a "trick" IMHO so I'd be curious how frequent that is : text!
What I mean is I know of Dark Souls but I haven't played it. Yet, solely by putting the only visible piece of text available (OK let's ignore "20" also) in a search engine "Asylum Demon" I get relevant suggestions, actually helpful content like fextralife right away.
I'd argue then the interesting question becomes how much context is needed to get useful advice... and more importantly how much context is gained with an image versus what the player already knows, e.g. game name, maybe "level" or anything unique, e.g. here boss name.
I tried a few more that do not include text and some that are even cropped:
::: spoiler AI generated content User: Is that weapon good?
Gemma 3 27b: Okay, let's analyze that weapon in the image and whether it's "good" in Dark Souls.
The Weapon: Estoc
The weapon the player is wielding in the image is an Estoc.
Is it good?
Yes, the Estoc is generally considered a very good weapon in Dark Souls, especially for its early/mid-game viability. Here's a breakdown of why:
However, there are some things to consider:
In the context of the image:
The player is at the beginning, so having an Estoc at this point is a great advantage. It will serve them well through a lot of the game.
Overall: The Estoc is a very strong and reliable weapon in Dark Souls, particularly appreciated for its speed, range, and poking ability. It's a solid choice, especially for new players learning the combat mechanics. :::
Good: It still knows that it's Dark Souls. Bad: That is not the Estoc, it's the Demon's Greataxe.
::: spoiler AI generated content User: Is that weapon good?
Gemma 3 27b: Okay, let's analyze that weapon!
That is the Ghru Dagger from Dark Souls.
Is it good? Yes, but it's very situational. Here's a breakdown:
Overall: The Ghru Dagger isn’t a beginner-friendly weapon. It's a specialized tool for those who understand how to maximize its toxic potential. If you're aiming for a build specifically designed around toxic, then it's excellent. Otherwise, there are better general-purpose weapons. :::
Surprisingly, it still knows that it is Dark Souls but once again the weapon is wrong. ChatGPT was also not able to identify the weapon correctly.
::: spoiler AI generated content User: What is this and what does it do?
Gemma 3 27b: That is a Bonfire from the Dark Souls series of video games (specifically, it appears to be from Dark Souls Remastered given the graphics).
Here's what it does:
Bonfires are crucially important to survival in the notoriously difficult Dark Souls games. They're safe havens in a dangerous world, but they also come with strategic considerations due to the enemy respawn mechanic. :::
Not bad actually. Even recognized that it is the Remastered version of Dark Souls 1, which is correct.
::: spoiler AI generated content User: What do I do here?
Gemma 3 27b: Okay, you're facing the Taurus Demon in Dark Souls! This is a classic early-game boss encounter. Here's a breakdown of what to do:
Understanding the Fight
Strategies to Beat Him
Specific Tips
In summary, your main goal is to lure the Taurus Demon onto the stairs and attack his belly. Focus on dodging, and be mindful of the falling rocks.
Good luck! Let me know if you get stuck or have more questions. Do you want me to elaborate on any of these points, or provide a strategy for a specific weapon build? :::
Nope, that is the Asylum Demon not the Taurus Demon. The description of the Taurus Demon is also completely wrong.
I also asked ChatGPT on the same picture:
::: spoiler AI generated content User: What do I do here?
ChatGPT 5: You are in the Asylum Demon boss fight from Dark Souls (tutorial area).
Options:
Do not fight yet. Turn left and run through the small door — visible under the torch on the left wall. This is the intended path. You’ll get a real weapon and shield before returning to kill the demon.
If you stay: You can damage it slightly, but with your current gear (broken sword) you’ll do almost nothing and likely die.
After escape: Proceed through the asylum, collect the Estus Flask and Undead Asylum F2 East Key, then return to the boss room by dropping from above for a plunging attack to start the real fight.
Go left now. :::
That's actually a pretty accurate description on what to do. You could work with that.
Thanks again. Well the first sentence started so good, correct game, neat,... but then wrong weapon... so totally pointless.
Again this can eventually be fixed. It's "just" a data problem, and that's exactly what models (and the entire infrastructure of data centers and researchers funded by VC money) excel at. So I think one can safely bet it will get there.
But... today, can one genuinely imagine playing Dark Souls (or any other game) without... knowing it? Like how does search for the wrong weapon and sometimes the right one help? How is that more convenient that picking a weapon up the searching manually for its name on desktop or mobile knowing with 99% certainty it will be the right one and advice will be genuine and relevant?
Absolutely not, but I would love to see someone do a challenge run where they play through the entire game just with Copilot. No thinking allowed.
lol, performance art. That'd be interesting. I'd watch too.
Plus 20K people did watch a fish play Pokemon.
Also now that I think about it, it shouldn't be too hard to feed a vision model a specific subset, e.g. https://darksouls.wiki.fextralife.com/Weapons of the visuals of all equipment and only then give advice. There is so much hierarchical information in there, e.g. one doesn't get an Elden Ring weapon in Dark Souls, or does not get an end of the game weapon (except with glitches) after 1h of play time, etc so it's possible to narrow the search space a lot.
I imagine a lot can be done with just few curated sources. Now... again (and I apologize for repeating myself so much while possibly sounding pedantic), why? Like what's the actual point?
Do you mind trying it with some random skyrim door puzzle or something like that? I imagine it's closer to what they expect people would use it for, but I don't imagine it handling any better.
The whole thing is specially useless if you consider that people are much less likely to try it on games that are old enough for the AI to have had data about it than on brand new games that they know nothing about.
Stolen from https://www.eurogamer.net/skyrim-golden-claw-door-puzzle-solution-bleak-falls-barrow :
Gemma 3 27b:
::: spoiler AI generated content User: Can you solve this puzzle for me?
Gemma 3 27b: Okay, this is the "Blessing of Akatosh" puzzle from The Elder Scrolls V: Skyrim in the Temple of Kynareth.
Here's the solution:
You need to activate the stones in the order representing the stages of a dragon's life cycle. From left to right, these represent:
So, the order you need to press the stones is: Left, Middle, Right.
Activating them in this order will open the pathway to the Blessing of Akatosh. Good luck! Let me know if you get stuck elsewhere. :::
Correct game, wrong puzzle. Did detect that it is a puzzle though.
ChatGPT:
::: spoiler AI generated content User: Can you solve this puzzle for me?
ChatGPT: This image shows the pillar puzzle from Bleak Falls Barrow in The Elder Scrolls V: Skyrim.
The correct order of symbols for this puzzle is:
Snake – Snake – Fish (Whale).
Explanation: If you look around the room before the lever, you’ll see a broken pillar lying on the ground above the gate. It shows the solution pattern in that exact order. Set the three rotating pillars to match:
Left pillar → Snake
Middle pillar → Snake
Right pillar → Fish (Whale)
Then pull the lever — the gate will open. :::
According to the Eurogamer article, that is correct.
That's great we really need to get the AIs playing the game for me so I'll have more time to spend on things I enjoy: like standing in the bread line.
You do know that basically every single online game that you play records all text and voice chat, right?
No it won't. It can only look at what is on the screen of the person using it.
It also makes no sense to use for multiplayer games.
pretty soon they're gonna put ai in ai
Maybe they should put some I into AI first.
"reasoning" in LLMs is practically AI in AI, since the AI prompts itself.
They're already doing that. It's like nested if statements but LLMs
While I do want to someday have AI pals while gaming, I refuse to have Microsoft be one of them.
I've just switched from Windows 10 to Windows 10 IoT LTSC (long term support channel), designed for IoT means there's no AI garbage, game bar and any other crap installed, and I get updates until 2032.
Quite happy with it, after a disastrous experience with Windows 11 that essentially killed my system.
And no, I can't switch to Linux due to company requirements. I work in finance and some of the banks we use operate by hardware tokens that only have Windows drivers.
such stupid thing to sell that spyware with, makes me think of earlier people "buying" gold and gems from natives for worthless baubles because they didnt know better about value of either. And even if it doesn't gather any information you care about someone taking, its still taking something valuable for no compensation > the training data and information in general that might be of use to someone at least and thus makes microsoft more money.
Online connection? So I remove the internet from the thing and it just doesn't work? Easy fix.
If you're going to air gap it, there are better windows operating systems than 11
It isn't air gapped, I just have a program that can block internet access to anything running. My internet is bad enough I can tell when something is using almost anything, and I've blocked most windows processes. It's just another check box to click
I hope microsoft is ready to train copilot on hours and hours of cloud meadow.
Everytime I hear the name "Game Bar" all I can think of is this.
G-mail was beta for like 15 years while billions of people used it. The "beta" label doesn't mean not ready for prod, hasn't for decades at this point.
As with everything else copilot related, this is optional. It can be completely disabled, and I'm pretty sure it's disabled by default. Mountain out of a molehill.