Apologies if this isn't the right place, but I have moved instances a few times, and in doing so I've accumulated a number of subscriptions from my old instances. I always used the "migrate" feature, and I have roughly 200 communities I'm subscribed to. Most are from various other instances that aren't my own. That being said, I have Lemmy open on "subscribed" and "scaled", as I found this gives me just what I want to see when I boot up the site.
But it is very slow, sometimes even timing out on Jerboa. I thought it was my instance, but I've had this issue on other instances as well. I'd imagine federating all my subscriptions would take some time, but then wouldn't "All" cause even more problems? I've tried to look into this to no avail.
Am I the only one? I'm asking this more out of curiosity than an actual solution.
Happens to me too. Jerboa frequently times out as well. I thought it was because my access point is running on potato hardware, but i guess it's a thing other places too.
Same, thought it was just me or my instance (feddit.nl), started after some update. Unsubscribed from a lot of communities and switching to limit posts per page helped a bit. Te-subscribing made it somewhat worse again. IIRC it is some instances or communities specifically.
If you remember which instances are the worst, please let me know. I still get time-outs more often than I'd like... Though it has become less after the server upgrade Ted did.
This is pretty much always going to be the case because the database query is much more complicated to filter by your subscriptions rather than the completely unfiltered views that are just sorted a certain way.
I don't have direct experience working in the lemmy codebase, but I do in piefed, and this is just an inherent limitation of how databases work. Basically, there is a big table in the db with all the posts that the instance knows about. As an example, let's say that you want to look at your Subscribed feed and sort it by New. First, the database sorts all the posts by newest first (this step is very quick). Then it will filter out all the entries that are not in a community that you are subscribed to (this step is a lot slower). So, views where the database doesn't need to do that filtering step are a lot faster than others. There are other things that slow down queries as well like blocks or keyword filters.
Smarter admins than I can employ strategies to tune their database for better performance. I don't have any knowledge in this area though, so others would be able to provide more insightful answers.
After some conversation back and forth, I was asked to continue the discussion here Context links: https://lemmy.ml/post/27659153/17534521 https://matrix.to/#/!tUnhsBRCsiePXfsIGe:matrix.org/$OufmGM...
Strictly speaking, the db might be looking in an index to choose rows by the communities — but using such a condition is pretty much guaranteed to be slower than not using it, anyway.
The actual answer depends on the actual database organization, of course. Ideally the whole database should be organized around frequent queries.
It also further links to another issue about individually blocking users and communities. Apparently that is quite inefficient in the current version, so maybe that adds to your problem?
Requirements Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support or the matrix chat. Did you check to see if this issue already exists? Is this only a single bug...
Every instance is like this it seems, maybe lemmy doesn't make a batched select at the cache and the db, but idk.
Maybe a new ~~webscale~~ backend could solve this, but i'm just a java backend developer, not a rust one. Maybe could be an interesting project, scala + pulsar + batching, turning lemmy into a microservices hellscape.
To add to other answers, the result for the ‘all’ feed is likely to be cached, either explicitly by the server app or implicitly by the database. Personal feeds are less likely to be cached, since they're only used by individual users.
This is probably true for anonymous (not logged in) users, but might not be if you are logged in. Things like user/community/instance blocks, votes, read/unread status, etc. need to be accounted for when generating the main feed for logged in users, and those are dynamic. It might be cached for some time, but the retention time would likely be short (a couple minutes maybe).
I haven't worked too much with the caching that piefed does, but I know it keeps some info in a redis cache to help speed things up (blocks, votes, etc.). That way getting all that info doesn't need to be a new db query every time except maybe the first time. Some pages also make use of etags so that if the content is unchanged, the request doesn't need to hit the db at all. I imagine lemmy has some mechanisms to do caching as well, but I am unfamiliar with the code at that level.
The HTTP ETag (entity tag) response header is an identifier for a specific version of a resource.
It lets caches be more efficient and save bandwidth, as a web server does not need to resend a full response if the content has not changed.
It's likely much faster to fetch the common feed from the database cache or prepared cache like Redis, and apply all this additional data in the app, than do uncached joins. So I'd hope that the apps do this. Especially since you say they use Redis, which of course doesn't do joins and such, unless something changed in the past years.
Just wanted to add "for me too". Sometimes my app takes a few minutes to load/time out the "subscribed" feed, while the "all" feed works fine and even my notification page has no issues.
Aside from general issues others have mentioned, our instance (slrpnk.net) is seeing some especially high database load in the last couple of days and I also noticed the subscribed page to be even slower than usual. I tried to figure out what it causing it, but so far there is no clear smoking gun, but I suspect some AI scrapers found a way to target the Lemmy API directly so our current scraper protections for the webinterface are inadequate.
What you can try is to clear your browser cache for the main domain. In the past there was a bug in Lemmy that caused Firefox based browsers to accumulate many gigabytes of cache data and that slowed down the loading of the page significantly. In the latest version there are some fixes for this and it shouldn't effect app usage, but I suspect this problem still persists to some extend.
Сын подготовил доклад о Тициане Вичелло. Сказал, что в книжке увидел его картину, ему понравилась. Понесет книгу, чтобы показать ее. Попросили показать картину. В итоге пойдет без иллюстраций, так как решили не рисковать, чисто на всякий случай. И да, это его единственная картина в этой книге.
За что люблю китайцев, так это за любовь к эргономике. Ту самую, что породила хентай, тентакли и прочие радости азиатских сайтов 18+. Иначе я никак не могу объяснить чем думал инженер располагая ввод прутка в принтер так, что средний китаец даже не сможет до него дотянуться, ему банально длины рук не хватит. Или предполагается что принтер будет украшением дома и будет стоять на постаменте посреди комнаты? Чтобы в лучах софитов, камеры и вот это всё...
@mstadmin ну это ещё европейцы придумали - поставить стартер в развал блока цилиндров 😀 чтобы для замены стартера надо было снять весь впуск как минимум 😀 Хотя на танке не V-образник, там наверняка подумать пришлось как бы так затолкать, чтобы наверняка 😀
Чуваки и чувакессы, инстанс обновлен с 3.5 до 4.4.8! Надеюсь меня видать вообще 😀 Я в процессе повидал некоторое дерьмо, но нейронка все разрешила и я просто копипастил команды, слава Омниссии это не отрывало меня от кины.
Just upgraded to MacOS 26.1 on a computer w/ an external 1920x1200 monitor and it looks TERRIBLE. The fonts in the official apps (Messages, etc) look grody with no way to change apparently. The whole interface is like an sophomore attempt w/ the over-rounded edges. So gross & amateurish #macos
We're thrilled to announce the next device in the Home Assistant Connect line! Join us on YouTube, November 19 @ 12:00PM Pacific / 3:00PM Eastern / 21:00 CET - the best is about to get better. 😎 youtube.com/live/HSRvso6cbf0 youtube.com/live/HSRvso6cbf0
I love photos like this. There’s nothing going on, really, someone just saw her and said “Oh she's SO PRETTY. I have to take a picture.” From my collection, 1959.
"I've listened to what was called "Country Rock" since early '70s and have been listening to BootLiquor for over 15 years now, and I'm STILL hearing/learning about bands & artists I've never heard of. Keep it going & I'll keep donating!" -- Sack
Вчера пошёл на прогулку, а там suntimes говорил, что луна не просто полная на 100%, но ещё и какое-то супер полнолуние
Ну луна как луна, пока её не закрыло облаками
Вот тут я протащился, я вообще не припоминаю чтобы видел такое. Не просто с таким градиентным ореолом, но даже на 11 часах виднеется сектор лунной радуги
Последние две недели небо вообще доставляет, я каждый вечер глазею во все зенки и балдю
🎯 Brand new target picker with full context 🤖 New way to add triggers, conditions, and actions 🏷️ Flexible entity naming in dashboard cards 🥧 Energy pie charts!
A brand new target picker with full context, redesigned dialog for adding automation blocks, flexible entity naming on dashboard cards, energy pie charts, and progress tracking for Home Assistant a...
Does anybody have suggestions for smaller/cheaper things to play with in HA?
I run proxmox at home, and am interested in this stuff but have been so anti-"connected" products I just don't even know what the product landscape is like.
I hate that the world has broken me so much that I read:
“PS: Oh, and pssst… Don’t tell anyone 🤫, but there might be something exciting being released on November 19th. Hit the bell on this announced YouTube stream to not miss it . Stay tuned! 😀”
And I immediately think, - Oh it's some AI bullshit, isn't it.
Maximum PC is the magazine that every computer fanatic, PC gamer or content creator must read. Each and every issue is packed with punishing product reviews, insightful and innovative how-to stories and the illuminating technical articles that enthu…
@rf Ну что же в целом сегодня день прошёл хорошо оценок просто куча причём хороших в принципе доволен что у меня всё получается нас одногруппниками как-то вообще не ладится все пытаются выехать на мне
прогрессивная голливудская звезда здорового человека. ехать в Херсон, где русня убивает гражданских всем чем возможно - от дронов до фабов, а не топить за хамас, сидя дома на диване за 20 штук баксов.
Pudutr0n
в ответ на srasmus • • •нравится это
srasmus, your_paranoid_neighbour, WheresTheFudge, sleepingoddish, Angel Mountain, Mac, Vicinus и humiddragonslayer нравится это.
paequ2
в ответ на srasmus • • •нравится это
srasmus, Pudutr0n, your_paranoid_neighbour, WheresTheFudge, Angel Mountain, Mac, Vicinus, humiddragonslayer и gerryflap нравится это.
Nightwatch Admin
в ответ на srasmus • • •нравится это
WheresTheFudge, Vicinus, humiddragonslayer и gerryflap нравится это.
Da Bald Eagul
в ответ на Nightwatch Admin • • •нравится это
humiddragonslayer, gerryflap и Nightwatch Admin нравится это.
Nightwatch Admin
в ответ на Da Bald Eagul • • •Da Bald Eagul нравится это.
wjs018
в ответ на srasmus • • •This is pretty much always going to be the case because the database query is much more complicated to filter by your subscriptions rather than the completely unfiltered views that are just sorted a certain way.
I don't have direct experience working in the lemmy codebase, but I do in piefed, and this is just an inherent limitation of how databases work. Basically, there is a big table in the db with all the posts that the instance knows about. As an example, let's say that you want to look at your Subscribed feed and sort it by New. First, the database sorts all the posts by newest first (this step is very quick). Then it will filter out all the entries that are not in a community that you are subscribed to (this step is a lot slower). So, views where the database doesn't need to do that filtering step are a lot faster than others. There are other things that slow down queries as well like blocks or keyword filters.
Smarter admins than I can employ strategies to tune their database for better performance. I don't have any knowledge in this area though, so others would be able to provide more insightful answers.
нравится это
undefined, Binette, srasmus, WheresTheFudge, sleepingoddish, frongt, Die4Ever, DABDA, bufalo1973, Goodlucksil, Angel Mountain, Blaze (he/him), Vicinus, DrCake, Da Bald Eagul, t҉̠̙ǵ̣̞̄ͪ͜x̸̱͚̳ͫ͐̑̈ͯͣ̚n̒͌҉͉̦̜̝ͅ, humiddragonslayer, Ludicrous0251 и OccasionallyFeralya нравится это.
srasmus
в ответ на wjs018 • • •Interesting. So when you query subscriptions, it actually queries everything and then filters it.
I decided to look at the github for Lemmy and found this issue that seems to reflect what youre saying.
I'm having flashbacks of avoiding table scans during my internship. 🙂
Tackling the Big, Bad `post_view` query
subversive-dev (GitHub)нравится это
WheresTheFudge, SorteKanin, bufalo1973, Vicinus, t҉̠̙ǵ̣̞̄ͪ͜x̸̱͚̳ͫ͐̑̈ͯͣ̚n̒͌҉͉̦̜̝ͅ, humiddragonslayer и Björn нравится это.
SlurpingPus
в ответ на srasmus • • •Strictly speaking, the db might be looking in an index to choose rows by the communities — but using such a condition is pretty much guaranteed to be slower than not using it, anyway.
The actual answer depends on the actual database organization, of course. Ideally the whole database should be organized around frequent queries.
нравится это
srasmus, Vicinus, DrCake и t҉̠̙ǵ̣̞̄ͪ͜x̸̱͚̳ͫ͐̑̈ͯͣ̚n̒͌҉͉̦̜̝ͅ нравится это.
poVoq
в ответ на srasmus • • •[Bug]: Blocking many communities renders Hot and New feeds unusable
Coolie2 (GitHub)нравится это
t҉̠̙ǵ̣̞̄ͪ͜x̸̱͚̳ͫ͐̑̈ͯͣ̚n̒͌҉͉̦̜̝ͅ, humiddragonslayer, srasmus и Five нравится это.
your_paranoid_neighbour
в ответ на srasmus • • •Every instance is like this it seems, maybe lemmy doesn't make a batched select at the cache and the db, but idk.
Maybe a new ~~webscale~~ backend could solve this, but i'm just a java backend developer, not a rust one. Maybe could be an interesting project, scala + pulsar + batching, turning lemmy into a microservices hellscape.
Edit: but
нравится это
srasmus, WheresTheFudge, sleepingoddish, Vicinus и humiddragonslayer нравится это.
SlurpingPus
в ответ на srasmus • • •нравится это
frongt, Die4Ever, DABDA, Vicinus, ragingHungryPanda, Da Bald Eagul, humiddragonslayer, Björn и gerryflap нравится это.
wjs018
в ответ на SlurpingPus • • •This is probably true for anonymous (not logged in) users, but might not be if you are logged in. Things like user/community/instance blocks, votes, read/unread status, etc. need to be accounted for when generating the main feed for logged in users, and those are dynamic. It might be cached for some time, but the retention time would likely be short (a couple minutes maybe).
I haven't worked too much with the caching that piefed does, but I know it keeps some info in a redis cache to help speed things up (blocks, votes, etc.). That way getting all that info doesn't need to be a new db query every time except maybe the first time. Some pages also make use of etags so that if the content is unchanged, the request doesn't need to hit the db at all. I imagine lemmy has some mechanisms to do caching as well, but I am unfamiliar with the code at that level.
ETag header - HTTP | MDN
developer.mozilla.orgSlurpingPus
в ответ на wjs018 • • •Avid Amoeba
в ответ на srasmus • • •нравится это
Vicinus и humiddragonslayer нравится это.
Angel Mountain
в ответ на srasmus • • •нравится это
Vicinus, t҉̠̙ǵ̣̞̄ͪ͜x̸̱͚̳ͫ͐̑̈ͯͣ̚n̒͌҉͉̦̜̝ͅ, humiddragonslayer, ℕ𝕖𝕞𝕠 и srasmus нравится это.
poVoq
в ответ на srasmus • • •нравится это
t҉̠̙ǵ̣̞̄ͪ͜x̸̱͚̳ͫ͐̑̈ͯͣ̚n̒͌҉͉̦̜̝ͅ, humiddragonslayer, wizardbeard, ℕ𝕖𝕞𝕠, srasmus, gerryflap, northernlights, Five и ProdigalFrog нравится это.
poVoq
в ответ на srasmus • • •нравится это
t҉̠̙ǵ̣̞̄ͪ͜x̸̱͚̳ͫ͐̑̈ͯͣ̚n̒͌҉͉̦̜̝ͅ, humiddragonslayer, ℕ𝕖𝕞𝕠, srasmus, Five и ProdigalFrog нравится это.