So Home is a separate 1.8TB NVME drive... But under home is my home directory, and under that is a half-dozen NAS mounts, including my Plex stuff. collection of ISO images. ;-)
Yeah, a single raw blu-ray can be over 100GB. I ripped my whole doctor who blu-ray collection once, it took quite a few days. And that wasn't even 4k or HDR or anything. The first few seasons are even interlaced iirc.
That's a good deal. Microcenter was selling a WD 20TB external HDD for $600. Didn't end up pulling the trigger 'cause I'm going for a 2 drive Raid1 config on my janky setup and $1350 w/ taxes is way too steep.
SAS actually ends up being a little cheaper due to it not being as compatible with home systems... And it's a refurb but I got lucky because smartctl showed it only had about 200 hrs of uptime reported.
Meanwhile here I am trying to upgrade my 512gb NVME drive to 2Tb while also still trying to afford car payments, rent and food. Rookie numbers on my part.
There’s probably a bunch of permissions errors, filesystems warnings for cross-filesystem mounts or links, etc. all going to stderr. Linux output streams are a bit odd, 1 is stdout and 2 is stderr. So the command is redirecting the “noise” to null and just printing the actual command output. That would be my assessment, but OP could probably give a more correct answer..!
I create videos, and back up all of my raw footage. I make weekly videos, and the size ranges from 50GB up to 500GB or more. I have 105TB available, 90TB used at the moment. I also have a fully redundant set of another 105TB. My employer has unfortunately made it very easy to justify hoarding, as they'll sell me reputable used commercial drives for $10/TB.
The video archives are 53TB
TubeArchivist is 19TB
Legally acquired movies and TV is 10TB
Immich is 2TB
Those are the main users of data. A bunch of other folders are using anywhere from a gig to 500GB, but those are basically rounding errors.
I'm in Virginia - I love Washington state, spent some time in Issaquah a while back in the SeaTac area... But the last time we moved my wife told me in no uncertain terms "If you take another out of state job, you're going alone." (Too many years of travelling for work...)
I don't need a raw video older than a day after the video is finished. I also don't need TBs of legally acquired content. I also don't need TBs of archived YouTube videos. I don't really need a NAS. I don't need a phone. We don't really need any of this tech stuff actually.
To answer your question more seriously, they're nice to have sometimes, instead of having to re-edit finished videos to make a compilation or something. I'm also hoping that maybe I'll see some success some day and be able to hire a more skilled editor (or even just me from the future with improved skills) to turn old media into feature-length films or just better versions of what I released. I dunno. It's data hoarding, but when I do want it, it's super nice to have.
I've also seen multiple professional creators talk about regretting not keeping the original footage from their old videos, so I'm not making that mistake. It's just nice to have it if you ever do want it, and I have the skills to archive it myself on the cheap, as compared to paying something like BackBlaze $7/TB/mo.
Just checked and my average video this year is currently at 400GB, so that's $4*2 for redundancy, so $8 per week aka $35/mo. If it were in BackBlaze, my subscription fee would go up by $12 per month forever, and it'd currently be at $630/mo, with zero redundancy. When I frame it like that, I'd be a fool not to do it!
I have 120TB in a dell T630 at a condo i rent on the other side of the state... I replicate some Truenas volumes and proxmox backups to over VPN..an "in case the house burns down" kind of thing..
Yeah, I've considered doing that, but most of my media is static and doesn't change and, with my upload speeds, would take literal days to sync. So I just have a set of HDDs that I keep across the state. I'll loose some if everything fails, but at least I'll have most of it.
I brought the two into the same room for the initial sync...then drove the T630 to the condo and hooked up, and it's all incremental updates over the 1G VPN. :)
Yeah, I have 40Mbps uploads at my house, that's the best I can get. I create anywhere from 100GB to 3TB of footage per week. On the high end, it'd take an entire week to sync, sucking up all of my upload bandwidth for that time, meaning I wouldn't even be able to upload the videos I create to YouTube/PeerTube in a reasonable amount of time. When I get faster upload speeds I'll definitely have to build a remote NAS for closer to real time backups.
OOhh ouch, that sucks ass.... If I didn't have Gigabit synchronous at the source there is no way I would even attempt replciation...I'd be carrying spare drives with me every time I go down to visit my grandson. ;-)
81 replies
I was like "nothing wrong here" until I saw that T that my brain just refused to parse the first time.
If I was on my laptop and not my phone I would post a screenshot with a P just for you
lol , is home separate mount point?
Yeah,
So Home is a separate 1.8TB NVME drive... But under home is my home directory, and under that is a half-dozen NAS mounts, including my
Plex stuff.collection of ISO images. ;-)omg. how many isos to get 100tb
If you're doing raw bluray, not as many as you'd think.
Yeah, a single raw blu-ray can be over 100GB. I ripped my whole doctor who blu-ray collection once, it took quite a few days. And that wasn't even 4k or HDR or anything. The first few seasons are even interlaced iirc.
BR is 25/50G, 4K BR is 66/100G
Uncompressed raw blu-ray rips most certainly are not.
Your username absolutely does not check out. Or your shredder is broken haha
I shred paper. ;-) After digitizing it of course. ;-)
For working I'm a backup and DR guy...the name was intended to be ironic. ;-)
Pfft, the only "hoarding problem" is that storage is expensive these days!
I know, I just paid $500 for a 24TB SAS drive that was $250 just over a year ago.
That's a good deal. Microcenter was selling a WD 20TB external HDD for $600. Didn't end up pulling the trigger 'cause I'm going for a 2 drive Raid1 config on my janky setup and $1350 w/ taxes is way too steep.
SAS actually ends up being a little cheaper due to it not being as compatible with home systems... And it's a refurb but I got lucky because smartctl showed it only had about 200 hrs of uptime reported.
I'm reading the command to the tune of Du Hast
Du hast mich. Du hast mich gefragt UND ICH HAB SPEICHERPLATZ
Toller Musik Geschmack!
Meanwhile here I am trying to upgrade my 512gb NVME drive to 2Tb while also still trying to afford car payments, rent and food. Rookie numbers on my part.
The most well-timed thing I ever did was buy 6 2tb NVMe drives in August last year
God help me if one fails
ZFS it
That's why there are 6. There's a pool with a hot spare. Just don't want to pay $350 or whatever to replace a $90 drive.
Pardon my stupidity BUT why include stdout to Devnull? Why not omit and simply 'du -sh /home'
There’s probably a bunch of permissions errors, filesystems warnings for cross-filesystem mounts or links, etc. all going to stderr. Linux output streams are a bit odd, 1 is stdout and 2 is stderr. So the command is redirecting the “noise” to null and just printing the actual command output. That would be my assessment, but OP could probably give a more correct answer..!
Nope, you are exactly on.
Noob, just use sudo, less chars!
Oddly enough, still generates errors. (There are stuff in user directories that are set to 600.... so even root can't browse/open.)
Well now I have to try this. Missing that executable bit would make sense. But last time I did this on / I didn't get errors 🤔
Are you sure? Root can see everything.
On a network, it can't.
NFS mount probably.
There's also some stuff in Proc and run that won't let you.
2> means stderr.... Keeps the "can't access ...." Out of the display.
That's stderr, not stdout
One think I miss from reddit… /r/datahoarder
These were my people. I probably have 100TB but it certainly isn't in my home directory. I'm not sure if I should be immpressed or freightened.
There's ![email protected], ![email protected], and ![email protected]...
EDIT: spelling
Well TIL … I subscribed to the first two but the last one didn't work for some reason.
Looks like a typo: ![email protected]
Yeah I totally missed that.
I agree.
Not gonna lie... I have some space...
never saw the s argument and was curious what's the difference to d. man pages are way ahead of me ^^
So just did a couple of experiments...
sudo su -sh /home - returns permission denied errors on certain NAS subdirectories, but not a lot.
du -sh /home --summarize -returns the same errors.
du -sh --max-depth=0 - returns the same errors plus an error saying that using --max-depth=0 is the same as --summarize.
;-) for the purposes of what I was doing (creating a clip for posting) redirecting stderr to null was the best option.
But I learned a few things today, which is cool. ;-)
that is always the most important. thanks for sharing!
Now I need to try that.
Greetings, fellow data hoarder!
Why do you pipe stderr of du into /dev/null?
To keep the errors out and provide just the result.
Jesus...
I went there once, on Reddit, and they are indeed over PB for some years now.
I myself already have 50T and it feels like a bottomless pit
I'm at single digit TBs and dreading that a drive breaks and I need a new one. Not that I have a lot of free space either.
Bruh
To be fair, all but about 2TB of that is on a NAS...but still.
I create videos, and back up all of my raw footage. I make weekly videos, and the size ranges from 50GB up to 500GB or more. I have 105TB available, 90TB used at the moment. I also have a fully redundant set of another 105TB. My employer has unfortunately made it very easy to justify hoarding, as they'll sell me reputable used commercial drives for $10/TB.
The video archives are 53TB
TubeArchivist is 19TB
Legally acquired movies and TV is 10TB
Immich is 2TB
Those are the main users of data. A bunch of other folders are using anywhere from a gig to 500GB, but those are basically rounding errors.
Ask you're employer if they're hiring...or..you know...adopting. ;-)
Are you living in or willing to move to Spokane, WA? I will say, I might be bias, but it is nice here.
I'm in Virginia - I love Washington state, spent some time in Issaquah a while back in the SeaTac area... But the last time we moved my wife told me in no uncertain terms "If you take another out of state job, you're going alone." (Too many years of travelling for work...)
But I'm an awesome remote worker. ;-)
How often did you need a raw video older than one year?
I don't need a raw video older than a day after the video is finished. I also don't need TBs of legally acquired content. I also don't need TBs of archived YouTube videos. I don't really need a NAS. I don't need a phone. We don't really need any of this tech stuff actually.
To answer your question more seriously, they're nice to have sometimes, instead of having to re-edit finished videos to make a compilation or something. I'm also hoping that maybe I'll see some success some day and be able to hire a more skilled editor (or even just me from the future with improved skills) to turn old media into feature-length films or just better versions of what I released. I dunno. It's data hoarding, but when I do want it, it's super nice to have.
I've also seen multiple professional creators talk about regretting not keeping the original footage from their old videos, so I'm not making that mistake. It's just nice to have it if you ever do want it, and I have the skills to archive it myself on the cheap, as compared to paying something like BackBlaze $7/TB/mo.
Just checked and my average video this year is currently at 400GB, so that's $4*2 for redundancy, so $8 per week aka $35/mo. If it were in BackBlaze, my subscription fee would go up by $12 per month forever, and it'd currently be at $630/mo, with zero redundancy. When I frame it like that, I'd be a fool not to do it!
I have 120TB in a dell T630 at a condo i rent on the other side of the state... I replicate some Truenas volumes and proxmox backups to over VPN..an "in case the house burns down" kind of thing..
Yeah, I've considered doing that, but most of my media is static and doesn't change and, with my upload speeds, would take literal days to sync. So I just have a set of HDDs that I keep across the state. I'll loose some if everything fails, but at least I'll have most of it.
I brought the two into the same room for the initial sync...then drove the T630 to the condo and hooked up, and it's all incremental updates over the 1G VPN. :)
Yeah, I have 40Mbps uploads at my house, that's the best I can get. I create anywhere from 100GB to 3TB of footage per week. On the high end, it'd take an entire week to sync, sucking up all of my upload bandwidth for that time, meaning I wouldn't even be able to upload the videos I create to YouTube/PeerTube in a reasonable amount of time. When I get faster upload speeds I'll definitely have to build a remote NAS for closer to real time backups.
OOhh ouch, that sucks ass.... If I didn't have Gigabit synchronous at the source there is no way I would even attempt replciation...I'd be carrying spare drives with me every time I go down to visit my grandson. ;-)
That's exactly my plan haha, and yeah, it sucks. Thanks Comcast.
Bob Saget would roll in his grave if he heard about your addiction.
"You ever suck some dick for some HDDs?"
Lately, I'm considering it.
That's an option?
Is there any advantage of using
duoverdffor this?Someone may eventually write CLI programs called
hastandmichthat you can somehow usefully pipe to itNow that’s a good idea
dfonly shows partitions, whereasduadds up the file sizes in the directory you specify.So, in particular, if you want to find out what's taking up so much space, you can repeatedly run
du -sh *andcdinto the largest directory.What he said. DF won't take into account the contents of mount points within a directory.
Or you run
ncduand save some keystrokes.What he said. DF won't take into account the contents of mount points within a directory.
What he said. DF won't take into account the contents of mount points within a directory.
What he said. DF won't take into account the contents of mount points within a directory.
Yeah it lets you have time to get some tea while it works.
I gotten used to
dustinstead ofdubecause it has much nicer visual representation.Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:
6 acronyms in this thread; the most compressed thread commented on today has 9 acronyms.
[Thread #35 for this comm, first seen 1st Jul 2026, 05:50] [FAQ] [Full list] [Contact] [Source code]