There are very few changes between rc1 and rc4 on lemmy_server, I see nothing that would deal with backend PostgreSQL performance issues:

lemm.ee has upgraded to rc4 in production, it has almost as many comments stored in the database as Lemmy.world