Queue-based file system for cache
I can't seem to find anything like it, neither as a Fuse fs. Is there really no file system, that just evicts the oldest entries once it's full? I mean, it could be a simple queue.
This would have been the ideal tool for cache; no bothering with size limits and background daemons.
10 replies
You probably can't find it because you're mixing mutually exclusive terminology.
A stack is a First-In Last-Out data structure. If your entry criteria is time, the oldest files will persist and the newer ones will get purged first.
If you want to remove oldest files first, you want a First In First Out queue.
Both of these will permanently delete the original file.
Caches are just a copy of real data, sitting somewhere that ideally has faster access times but smaller storage space. The real original file does not get modified. A cache will typically hold onto as much data as it can until it hits some cap, at which point a replacement algorithm is used to select which entry or entries to make space for the newly requested item. Very rarely will a cache delete entries on its own to minimize actively used space, but if they do they will likely just run the replacement algorithm without backfilling the space
Right, yeah, queue, not stack, thanks.
About the last sentence, that's literally the freedesktop definition that cache can get deleted at any time. Important runtime data is state data.
A queue is imho ideal, because only the oldest cache no process cares anymore, gets purged.
Not 100% on what 'stack-based' means here, but if 'stacking' = 'tiering', you could use a normal filesystem like bcachefs that have tiered storage ?
You can declare different tier devices for speed, availability or what you prefer, and have different compression, managed cache, auto-move hot data from cold storage, cold data to cold storage, and what not..
However, I've never actually tried it, so I'm likely exaggerating the possibilities. I think ZFS have tiering too, but maybe a filesystem specialist can chime in with some advice.
Alternatively, if it doesn't bother you, it could be a great task for a sota ai, to help/create/setup a simple fuse fs for your needs.
Something like this? http://www.rrdtool.org/
Not at all. Is there a misunderstanding?
I guess I was thinking about it too abstractly.
Still, thanks.
What are you trying to build/do?
Uncomplicated cache/trash management
But sadly, doesn't seem to exist. Maybe i will have to learn how to create a fs? It's only a stack, would also be fast...
Sounds like you want to automate Trash folder management. Why would number 1 require complex rules if number 3 can work with only the rule of delete the oldest file? Wouldn't the same rule work in both cases?