I mean, we can take a very simple design idea to test this:
For your first quest, you do a fetch quest. But it turns out, gamers hate fetch quests!
We know such a result from colloquial knowledge and hundreds of hours spent watching armchair design essays. But game designers don’t have hundreds of hours, but they want hundreds of answers, based on more than video whims. Even indie devs (you can see a lot of them in NextFest) may make really bad design decisions for their little games.
In the HL2 commentaries, you see Valve gets tons of in-person QA feedback at their Seattle office. That’s great if they have the resources to arrange it. Take me though. My testing is “please read my book - no controller required” and that’s astoundingly hard to push on random people, to the point some would consider paying individual players/testers/readers.
From that standpoint, collecting metrics doesn’t seem so unreasonable. I get that data collection is an evil and ruthless business at its extreme end, but “How many players found this menu” just seems mutually beneficial to know.