So I've been working on a solution for this.

As I see it Google and others are going to have a hard if not impossible time to incorporate the fediverse, and the fact that the same content can exist on multiple servers.

So I'm working on a search engine specifically build, for Lemmy at least. Where it'll take you to whatever your preferred instance is when tapping on a search result.

I hope to have a MVP up and running in a few more days.

I don't mean to keep self promoting but: https://github.com/marsara9/lemmy-search

It's still a work in progress, but it's going to be a search engine specifically built for Lemmy, and maybe eventually the entire fediverse.

Idea is that you'll be able to search just like you normally would in Google but

1. the links it shows you would take you to your preferred instance
2. it will search as many lemmy instances as it can find, so it won't find anything outside of the fediverse.

I'm working on a specialized search engine just for the fediverse. https://github.com/marsara9/lemmy-search

If anyone wants to help out, feel free to reach out, but I hope to have something ready to release soon.

The idea with my version is that it'll search as much of Lemmy / the fediverse as it can and you can select the preferred instance that you want to open any link with.

Short answer -- nothing.

Longer answer -- Those communities, posts and comments will still exist on the other servers, but sadly interacting with them will not federate that data to other instances anymore, since the host instance is no longer active.

IMHO federation doesn't bring any real benefits to git and introduces a lot of risks.

The git protocol, if you will, already allows developers to backup and move their repositories as needed. And the primary concern with source control is having a stable and secure place to host it. GitHub already provides that, free of charge.

Introducing federation, how do you control who can and cannot make changes to your codebase? How do you ensure you maintain access if a server goes down?

So while it's nice that you can self host and federate git with GitLab, what value does that provide over the status quo? And how do those benefits outweigh the risks outlined above?

Playing devil's advocate for a bit... So these are just cross-posts. Which existed even on Reddit. ...I assume they weren't handled in any way in Sync or Reddit?

But let's say this is fixed... What to do about the multiple comments threads? How would you reconcile them with each other? Especially since the user can choose different ways to sort the comments as well. Would all of this logic normally handled by the Lemmy back-end now need to run on your phone? Also how do you choose which post / instance to actually display and which ones to hide?

Btw, I'm not trying to dismiss the idea. Just want to call out some of the technical problems that might come up trying to implement such a feature. As well as ask questions to try and determine exactly how such a feature is expected to work.

Thanks for the shout-out.

But FYI I've run into some bugs that's preventing new content from being indexed. So you won't see anything new (from about a week ago) until I can find a new method to fetch new posts.

Checking myself because you got me curious: https://github.com/LemmyNet/lemmy/blob/main/crates/api/src/local_user/login.rs

But passwords are hashed. Specifically using bcrypt: https://en.m.wikipedia.org/wiki/Bcrypt

Yes but that search doesn't take you to the instance that you are logged into already. Which is one of my main goals with this site. While that did give me the inspiration for this and has the power of Google behind it, it lacks knowledge about how the fediverse actually works.

I'm doing tests in the next couple days. But I'm trying to build a search engine specifically for Lemmy.

• It should in theory work similar-ish to Google / Bing.
• You can filter by instance, community or author.
• it only indexes Lemmy posts and it won't keep duplicates.
• It'll also open any link you find in your instance.
• You'll be able to self host it and point it to any instance you want as well.

I'm hoping I can open it to the public in a week or so.

I can't speak for anyone else but I can at least share why I didn't care for it.

Steam already exists and there isn't a lot I feel is missing from the Steam store. Not to mention there isn't anything that Epic does better than Valve for their storefronts. Epic doesn't provide any new value that Valve doesn't. In fact I'd argue that Epic causes negative value for several reasons:

1. They essentially fragmented the PC market as now I can't access all of my games in a single location. The same reason for example that I hate that Netflix, Hulu, Disney+, HBO Max, etc ... exist. It just makes the overall experience less convenient and more expensive.

2. Several features that Steam already has working are either broken or missing in the Epic Store. Especially around the shopping cart, etc ... I haven't gone to look recently to verify if any of these have been fixed and probably won't until the other points are addressed, if they even can be resolved.

3. There's no level of trust with Epic compared to Steam. I have a massive Steam collection and Valve has shown time and time again that I can trust that my data is relatively safe with them. When was the last time you saw Valve in the media because of a data breach? When was the last time your Steam account got hacked. Epic is just new here so they haven't had a chance to earn that trust.

4. Their motives for wanting to create a new store wasn't to improve the customer experience. Instead it was to improve their bottom line. The court cases against Google and Apple prove this. If they at least tried to have a PR campaign to show how Epic is innovating compared to Steam especially for making the customer experience better, the Epic Store might have sit better with me.

5. Timed exclusives. Similar to point 1, but they were just trying to use their massive cash reserves from Fortnight to buy their way into the market rather than earn the trust of customers. This again resulted in fragmentation of the PC market.

Anyway, just my two cents.

Yep and I'm one of them. Go look me up on Reddit and I think I have maybe 20 posts over the 14+ years I was on the site. ...joined Lemmy and immediately got frustrated that I couldn't find anything. So I figured I take a crack at it. Especially since I couldn't see how Google would ever be able to link me to my instance. Let alone make it easy to search the entire fediverse without having to write out every possible site, with new ones popping up every day.

With ActivityPub all of the primary ids contain the domain of the hosting server. So if you lose your domain none of the other instances know that you're the authority on those communities, posts, comments or users. So essentially federation breaks with all of the old data.

Thanks. If you do some digging you can find the project on GitHub but note that it's a work in progress still. The UI is lacking and it's rough around the edges but it's "working". And I still need to do some optimizations on the crawler itself, etc....

It's also going to be completely self-hostable just like Lemmy, etc...

https://www.search-lemmy.com

http://www.github.com/marsara9/lemmy-search

Just add community:[email protected] at the end of your query.

Not yet but I can add this feature

Lemmy is entirely open source, so you can see what their architecture looks like, etc.. here: https://github.com/LemmyNet/lemmy.

Rate limits, as I understand them from the code, should only apply on a per-IP basis. So you should only be seeing rate limit errors if:

• your behind a CGNAT and multiple people who use your ISP are using Lemmy
• you're sending A LOT of requests to your instance yourself
• the admin of your instance has significantly lowered the rate limits (viewable here: /api/v3/site)

Sadly not yet. I'm in desperate need of a fronted dev, as the HTML/Javascript needs some serious work. But if I can't find someone soon I'll see what I can get put together and get it up and running soon enough.

Checkout the conversation here: https://lemmy.world/post/670532. This topic keeps coming up, so I'm hoping the solution I'm working on can resolve this exact issue.

I've already got some complaints about that. You can see one of the issues raised on GitHub.

At the moment, I'm only picking up mastodon posts that are federated to Lemmy, but you can't choose Mastodon as a preferred-instance, yet. When and/if I decide to add Mastodon support, I'll reach out to the admins over there to get feedback first.

Edit and note to any server admin: If you want to block the crawler from hitting your site, just add lemmy-search to your robots.txt and crawling will be prevented. But this doesn't stop cross-federation posts from being picked up on another instance.