replacing mlmym with a fork | Spyke

lemmytoday·LemmyTodaybyhexagonwin

replacing mlmym with a fork

hello. i'm a happy user of lemmy.today.

i've always used the mlmym frontend available on old.lemmy.today, it generally works well enough. however, there are some minor minor bugs and missing features. (e.g. impossible to navigate to next page when viewing saved posts, impossible to see down/upvotes separated)

i recently came across this fork of mlmym (from this github issue) and it seems to have all those bugs fixed. can we replace the current mlmym with this fork?

thanks for reading

replacing mlmym with a fork

https://code.mschae23.de/mschae23/mlmymOpen link View original on lemmy.today

7

discuss.mschae23.de

Hello! I'm the person who made that fork.

If it does get set up here and you have any questions or run into any issues (which could very well happen, as I've done some major refactors), feel free to contact me, either by email at [email protected] or in the #mlmym:mschae23.de matrix room.

You can also try it out already at discuss.mschae23.de (only logged-out here, of course, which doesn't give you all options) or by running it up locally with LEMMY_DOMAIN set to lemmy.today.

I'd also recommend enabling some of the options instance-wide by default, especially SHOW_UNREAD_COMMENTS (this one has been really useful for me).

2

mrmanager reply

Thank you for your work on this. We are running it now on this instance. :)

2

mrmanager reply

I wanted to ask you also if you could forward certain http headers to lemmy when it makes a request? Currently it doesnt forward them so lemmy thinks all the requests from mlmym to lemmy are from the docker ip its running on. And that makes it much harder to rate limit or ban bots since all requests are from the docker ip.

So basically, mlmym should preserve and forward X-Real-IP, X-Forwarded-For, X-Forwarded-Proto, and X-Forwarded-Host when proxying API/backend requests to Lemmy, because Lemmy uses these headers for correct client IP detection and rate limiting.

Something like this in the code where the incoming request is handled:

const forwardedFor = request.headers.get("x-forwarded-for");
const realIp =
  request.headers.get("x-real-ip") ??
  forwardedFor?.split(",")[0]?.trim();

const headers = new Headers(request.headers);

if (forwardedFor) {
  headers.set("x-forwarded-for", forwardedFor);
}

if (realIp) {
  headers.set("x-real-ip", realIp);
}

headers.set("x-forwarded-proto", request.headers.get("x-forwarded-proto") ?? "https");
headers.set("x-forwarded-host", request.headers.get("host") ?? "");

That would really help a lot and allow me and others to remove a lot of complicated workarounds for trying to get the source ip.

2

mschae reply

discuss.mschae23.de

I have some handling for X-Forwarded-For, but none of the others. This could actually explain why I was still seeing rate limit errors despite not having made many requests from my IP address.

I'm unfortunately busy this week, but I hope I can still find some time in the next few days to implement this.

headers.set("x-forwarded-proto", request.headers.get("x-forwarded-proto") ?? "https");

Shouldn't that fallback be http, since mlmym doesn't do HTTPS itself? And I assume the reverse proxy will have set X-Forwarded-Proto to https.

1

mrmanager reply

Yeah, I have a custom setup currently just to try and get the source ip, but it doesn't work properly in all cases. And most of the bots use old.lemmy.today since they know mlmym doesn't forward the source ip address, and therefore it's hard to rate limit them.

So it would be amazing if you could forward those 4 headers. Then I will setup nginx to forward those headers to mlmym, and mlmym will forward them when making a request to Lemmy.

I actually thought it was making a https connection to Lemmy but perhaps it doesn't, and nginx just forwards http to https. Not sure. Will have to look into this more.

But yeah, whenever you have time to add the headers, i will investigate more. Not super urgent but will be interesting to play with that later. And it will protect all Lemmy instances that use your fork, so thats good.

2

mschae reply

discuss.mschae23.de

Then I will setup nginx to forward those headers to mlmym

If you configure it to set X-Forwarded-For, you should have at least that information now already.

I actually thought it was making a https connection to Lemmy but perhaps it doesn't

That depends on the setup. If you haven't already, you should definitely set LEMMY_INTERNAL_URL so that mlmym connects to lemmy directly without TLS or a reverse proxy, which should be a lot faster (but that's only possible if they're running on the same machine, of course).

X-Forwarded-Proto is supposed to say what protocol the user (of mlmym) originally connected with though, right? Which will still end up being https in the common case of it being behind a reverse proxy that does TLS.

2

mrmanager reply

I ended up using the external nginx for setting rules for rate limiting and such, and there I see the real ip. It's before traffic even gets to mlmym so that's good.

Also made sure to make it use the internal Lemmy url and skip a lot of the proxy stuff I was using earlier. It seems faster now.

2

mschae reply

discuss.mschae23.de

I implemented the headers today, so version 26.2.1 should hopefully do this correctly now.

I could only really test it in my deployment, so feel free to let me know if you have any issues!

2

mrmanager reply

Cool. :) I think that will be a very useful feature for a lot of Lemmy admins.

Will play around with it later...!

2

mrmanager reply

I will experiment more with this tomorrow or the coming days then. I was trying to get it to work previously but failed (not using your fork but the original mlmym).

2

Yes, let's do it.

And sorry for the late response, I didn't see this post until I got a private message about it.

Will try and get it up today.

Edit: And its up. I enabled the SHOW_UNREAD_COMMENTS option as well.

2

hexagonwin reply

thanks a lot! it seems to work well :)

edit: for some reason the communities bar on the top doesn't seem to work properly :( before there was a list of all subscripbed communities, now it's blank..

2

mrmanager reply

Hmm, that's not good... Could be a bug in this version perhaps... Will have to roll back if I can't fix it somehow.

2

hexagonwin reply

i tried compiling it locally and there's the same issue. however it seems to appear in certain conditions but disappear again on a refresh..

now that we've rolled back, i'll let you know again if this bug gets solved.

thanks a lot

3

mrmanager reply

Perhaps it works now for you. I made some changes to the networking that was causing it to run slow, perhaps that also created your issues. If you hold shift and click reload in the browser to skip the browser cache, perhaps that will also work.

Is it the top bar in this picture you dont see?

2

hexagonwin reply

yes, the top bar (technology lemmyshitpost memes me_irl ...). it seems be working now for some reason, thanks

edit: hmm now comment scores aren't showing lmao. maybe there's something wrong with my browser?... seems like something's buggy...

2

mschae reply

discuss.mschae23.de

hmm now comment scores aren't showing lmao. maybe there's something wrong with my browser?... seems like something's buggy...

That looks like you have “show scores” disabled in the preferences? Can't think of any other reason they wouldn't be showing like this.

Also, if you're wondering about the “also show on comments” option for upvote/downvote totals, I found that too distracting to show on every comment, so that extra information is only shown when there are actually both upvotes and downvotes.

1

hexagonwin reply

That looks like you have “show scores” disabled in the preferences? Can't think of any other reason they wouldn't be showing like this.

"show scores" was already enabled, but disabling -> saving -> enabling fixed it, i guess the settings for my account was in a weird state. ~~it's all resolved now, thanks!~~

ok seems like i said that too early, the top bar disappearing issue came back, lol. it seems to work a few times randomly.. it isn't a critical bug so i'll keep using and investigating.

the 'my communities' button always seems to work

2

mrmanager reply

Ok see if you can report it to the author who posted in this thread. Once its fixed we can make another attempt. :)

2

mschae reply

discuss.mschae23.de

Hm, it's working on my instance. I did change it to only make the search request for top communities once and cache it, so maybe that happened to run into an error and it's not trying again?

Running it locally on lemmy.today, it does seem to be working. I don't have an account here, unfortunately, so I can't test whether the “my communities” dropdown works too, but I don't see why it shouldn't.

1

mrmanager reply

There could have been another issue with traffic being blocked.

Will try again.

2

mschae reply

Oh, while you're here, I think LANGUAGE_TAGS is also a good candidate to turn on by default (forgot to add it to the readme when I added the feature, so I also forgot about it in my earlier message...), and maybe SHOW_CONTROVERSIAL.

There's also EMBED_METADATA for embeds (for links on Discord and such) to show, but I don't know if you want that.

2

mrmanager reply

I enabled all of them. :)

2