google.com/goto: Google's anti-scraping update
https://www.autom.dev/blog/google-search-goto-linksOpen linkView original on programming.devSyndicated from the fediverse. Read and engage on the original instance.
View original on programming.devhttps://www.autom.dev/blog/google-search-goto-linksOpen linkView original on programming.dev
9 replies
…but that's how you follow a redirect?
You're right, and this article is spouting nonsense. If you have to request the
/gotolink in order to get the URL then you've already been tracked by Google (or blocked if they think you're a scraper), and following theLocationheader is exactly what your browser would do anyway. That's how HTTP redirects work.This article is from a company that offers APIs to just proxy Google search results, so they're exactly the kind of people that Google is trying to block with this change. Reading some other posts on their site (eg. the one that explains how to follow the Location header) there are obvious signs of slop, so this whole article is probably an AI hallucination.
One is under your control. The other is being passed through a script on the server that may be doing who-knows-what in addition to redirecting you.
(Better if you just drop Google altogether.)
No? If you want to resolve this URI, you have to follow the redirect, there's no way around it. Requesting it from google and looking at the
Locationheader is following the redirect.I mean, I agree that one should drop google and that this is probably just another thing they do to track people, but that's not what I meant with that comment.
Can someone just "distillation attack" google search so I can just use the derived data instead of giving them all my click tracking?
No. But you can use alternative search engines, or frontends like searxng that won't send your clicks to Google (while they still work, anyway, this change might break them)
I'd rather just have the original dataset.
That's the thing they've built the moat around.
You could build a browser extension to extract search results but the infrastructure will probably be expensive.
Eh. DDG and other search engines exist, and all Google has gained is that the entire rest of the search world (as well as button makers, context link creators, RSS, bookmarkers etc) have to drop them because the URLs don't actually resolve to results and are indistinguishable from phishing functionally speaking; meaning Google won't even get people's search results by proxy now.
Now, if all other big search providers do this and engines like DDG have to drop them, we'll reach a new golden age, where new and more ethical search providers can enter the playfield and start without a big disadvantage.