Zip makes different tradeoffs. Its compression is basically the same as gz, but you wouldn't know it from the file sizes.
Tar archives everything together, then compresses. The advantage is that there are more patterns available across all the files, so it can be compressed a lot more.
Zip compresses individual files, then archives. The individual files aren't going to be compressed as much because they aren't handling patterns between files. The advantages are that an error early in the file won't propagate to all the other files after it, and you can read a file in the middle without decompressing everything before it.
Yeah that's a rather important point that's conveniently left out too often. I routinely extract individual files out of large archives. Pretty easy and quick with zip, painfully slow and inefficient with (most) tarballs.
Voicebanks for Utau (free (as in beer, iirc) clone of Vocaloid) are primarily distributed as SHIFT-JIS encoded zips. For example, try downloading Yufu Sekka's voicebank: http://sekkayufu.web.fc2.com/ . If I try to unzip the "full set" zip, it produces a folder called РсЙ╠ГЖГtТPУ╞Й╣ГtГЛГZГbГgБi111025Бj. But unar detects the encoding and properly extracts it as 雪歌ユフ単独音フルセット（111025). I'm sure there's some flag you can pass to unzip to specify the encoding, but I like having unar handle it for me automatically.
I know inabakumori! Their music is so cool! When I first listened to rainy boots and lagtrain, it made me feel emotions I thought I had forgotten a long time ago... I wish my japanese was good enough to understand the lyrics without looking them up ._. I'm also a huge fan of Kikuo. His music is just something completely unique, not to mention his insane tuning. He makes Miku sing in ways I didn't think were possible lol
I get you, I want to learn more Japanese. I only understand a very small amount at this point. I don't have any Miku songs that I have really wanted to listen to, but that could change. I might check out Kikuo then. Also I love the animations Inabakumori release with their songs too. They have some new stuff that's really good if you haven't checked it out yet.
When I was on windows I just used 7zip for everything. Multi core decompress is so much better than Microsoft's slow single core nonsense from the 90s.
Makes sense. There are actual programmers working at facebook. Programmers want good tools and functionality. They also just want to make good/cool/fun products. I mean, check out this interview with a programmer from pornhub. The poor dude still has to use jquery, but is passionate to make the best product they can, like everone in programming.
There's several levels you can use to trade off additional space for requiring more processing power. That being said, I hate xz and it still feels slow AF every time I use it.
I usually suppress output of either wget (-q) or of tar (no v flag), otherwise I think the output gets mangled and looks funny (you see both download progress and files being extracted).
That's not Linux doing that. It's the demons in your hardware trying to escape. They normally don't cause too many issues luckily, but if you don't close the portals occasionally they can take over your system.
Yeah, those tend to be pre-folder settings for the File Explorer.
Like View options, thumbnails and such.
It's been a while for me, but I think there was something specially for thumbnails too.
You might find one if you go into the folder options and set a folder to optimized for pictures/videos and add some to it.
this is a complete uneducated guess from a relatively tech-illiterate guy, but could it contain mac-specific information about weird non-essential stuff like folder backgrounds and item placement on the no-grid view?
They're Metadata specific for Macs.
If you download a third party compression tool they'll probably have an option somewhere to exclude these from the zips but the default tool doesn't Afaik.
HFS+ has a different features set than NTFS or ext4, Apple elect to store metadata that way.
I would imagine modern FS like ZFS or btrfs could benefit from doing something similar but nobody has chosen to implement something like that in that way.
A Copy on White file system that supports snapshots, supported mostly by
ZFS
Zetabyte File System
Copy on Write File System. Less flexible than BTRFS but generally more robust and stable. Better compression in my experience than BTRFS. Out of Kernel Linux support and native FreeBSD.
HFS+
what Mac uses, I have no clue about this. some Copy on Write stuff.
NTFS
Windows File System
From what I know, no compression or COW
In my experience less stable than ext4/ZFS but maybe it's better nowadays.
Great summary, but I've to add that NTFS is WAY more stable than ext4 when it comes to hardware glitches and/or power failures. ZFS is obviously superior to both but overkill for most people, BTRFS should be a nice middle ground and now even NAS manufacturers like Synology are migrating ext4 into BTRFS.
Well that's good to know because I had some terrible luck with it about a decade ago. Although I don't think I would go back to windows, I just don't need it for work anymore and it's become far too complex.
I've also had pretty bad luck with BTRFS though, although it seems to have improved a lot in the past 3 years that I've been using it.
ZFS would be good but having to rebuild the kernel module is a pain in the ass because when it fails to build you're unbootable (on root). I also don't like how clones are dependant on parents, requires a lot of forethought when you're trying to create a reproducible build on eg Gentoo.
I'm the weird one in the room. I've been using 7z for the last 10-15 years and now .tar.zst, after finding out that ZStandard achieves higher compression than 7-Zip, even with 7-Zip in "best" mode, LZMA version 1, huge dictionary sizes and whatnot.
You can actually use Zstandard as your codec for 7z to get the benefits of better compression and a modern archive format! Downside is it's not a default codec so when someone else tries to open it they may be confused by it not working.
First bundling everything in a tar file just to compress the thing in an individual step is kinda stupid, though. Everything takes much longer because of that. If you don't need to preserve POSIX permissions, tar is pointless anyway.
Use an archiving format that does both at once then, preserving whatever tar use cases has and compressing. The two steps are stupid, no arguing against that.
I think it's a joke about how people from Poland have a lot of consonant letters in their name, particularly the letter Z. This appears strange to non-poles, and thus became the subject of many jokes. So ZSTD looks a little bit like a Polish name in the sense that it's made up of many consonant letters, including a Z.
More examples of people making fun of Polish names:
Zip is fine (I prefer 7z), until you want to preserve attributes like ownership and read/write/execute rights.
Some zip programs support saving unix attributes, other - do not. So when you download a zip file from the internet - it's always a gamble.
Tar + gzip/bz2/xz is more Linux-friendly in that regard.
Also, zip compresses each file separately and then collects all of them in one archive.
Tar collects all the files first, then you compress the tarball into an archive, which is more efficient and produces smaller size.
There’s gotta be a buncha tools that Clippy into the terminal to say “did you mean ____?” right? Including some new ones where they trained/fine-tuned a language model on man pages?
Interesting it’s not the most popular thing to use a GUI and use shortcuts for everything you want to do while still having the option to click through a menu or wizard for whatever you haven’t memorized. I suppose the power and speed of the command line are difficult to match if you introduce anything else, and if you spend time using a user interface that’s time you can’t spend honing your command line skills.
Zip, RAR, 7z etc. store and compress the files. Tarballs work differently, tar stores the files and the second program compresses the tar as a continuous stream.
They originated in different mediums, programs like zip were born to deal with folder structures, tar was created to deal with linear tape archives (hence the name).
Kind of redundant. Both .zip and .rar store an index of files within the archive and are a bit 'inside-out' when it comes what we get from tar.gz.
That is, ZIP is pretty close to what you'd get if you first gzipped all your files and then put them into a .tar.
RAR does a little more (if I remember correctly), such as generating a dictionary of common redundancies between files and then uses that knowledge to compress the files individually, but better. Something akin to a .tar file is still the result though.
If you download and extract the tarball as two separate steps instead of piping curl directly into tar xz (for gzip) / tar xj (for bz2) / tar xJ (for xz), are you even a Linux user?
They really, really aren’t. Let’s take a look at this command together:
curl -L [some url goes here] | tar -xz
Sorry the formatting's a bit messy, Lemmy's not having a good day today
This command will start to extract the tar file while it is being downloaded, saving both time (since you don’t have to wait for the entire file to finish downloading before you start the extraction) and disk space (since you don’t have to store the .tar file on disk, even temporarily).
Let’s break down what these scary-looking command line flags do. They aren’t so scary once you get used to them, though. We’re not scared of the command line. What are we, Windows users?
curl -L – tells curl to follow 3XX redirects (which it does not do by default – if the URL you paste into cURL is a URL that redirects (GitHub release links famously do), and you don’t specify -L, it’ll spit out the HTML of the redirect page, which browsers never normally show)
tar -x – eXtract the tar file (other tar “command” flags, of which you must specify exactly one, include -c for Creating a tar file, and -t for Testing a tar file (i.e. listing all of the filenames in it and making sure their checksums are okay))
tar -z – tells tar that its input is gzip compressed (the default is not compressed at all, which with tar is an option) – you can also use -j for bzip2 and -J for xz
tar -f which you may be familiar with but which we don’t use here – -f tells tar which file you want it to read from (or write to, if you’re creating a file). tar -xf somefile.tar will extract from somefile.tar. If you don’t specify -f at all, as we do here, tar will default to reading the file from stdin (or writing a tar file to stdout if you told it to create). tar -xf somefile.tar (or tar -xzf somefile.tar.gz if your file is gzipped) is exactly equivalent to cat somefile.tar.gz | tar -xz (or tar -xz < somefile.tar – why use cat to do something your shell has built-in?)
tar -v which you may be familiar with but which we don’t use here – tells tar to print each filename as it extracts the file. If you want to do this, you can, but I’d recommend telling curl to shut up so it doesn’t mess up the terminal trying to show download progress also: curl -L --silent [your URL] | tar -xvz (or -xzv, tar doesn’t care about the order)
You may have noticed also that in the first command I showed, I didn’t put a - in front of the arguments to tar. This is because the tar command is so old that it takes its arguments BSD style, and will interpret its first argument as a set of flags regardless of whether there’s a dash in front of them or not. tar -xz and tar xz are exactly equivalent. tar does not care.
Yes. You can specify tar -C somedir if you want it to extract them somewhere else.
As a rule of thumb, I always extract my tarballs in a newly created, empty directory, just in case whoever packed it didn't put all its files in a subdir
the problem is if the connection gets interrupted your progress is gone. you download to a file first and it gets interrupted, you just resume the download later
I stopped doing that because I found it painfully slow. And it was quicker to gzip and upload than to bzip2 and upload.
Of course, my hardware wasn't quite as good back then. I also learned to stop adding 'v' flag because the bottleneck was actually stdout! (At least when extracting).
Bzip2 compression is often surprisingly good with text files, especially log files. It seems to "see" redundancies there - and logs often have a lot of it - far better than gzip and sometimes even lzma.
Anyway, if I saw a bunch of tar.bz2 files, that's what I'd expect to find in them.
The two aren't really equivalent. They make different tradeoffs. The scheme of "compress individual files, then archive" from GP is what zip does. Tar does "archive first, then compress the whole thing".
The link does not load for some reason, but tar itself does not compress anything. Compression can (and usually is) applied afterwards, but that's an additional integration that is not part of Tape ARchive, as such.
I unironically used xz for a long time. It was just eazy and all around very good compress. A close second is 7zip because I used it on windows for years.
I still wonder what that's like. Somebody must still occasionally get a notification that SOMEWHERE somebody paid for their WinRAR license and is like "WOAH WE GOT ANOTHER ONE!"
Compatibility aside, I'd say that .tar.pxz aka .tpxz is probably my vote.
LZMA is probably what I'd want to use. xz and 7zip use that. It's a bit slow to compress, but it has good compression ratios, and it's faster to decompress than bzip2.
pixz permits for parallel LZMA compression/decompression. On present-day processors with a lot of cores, that's desirable.
Oblig. XKCD:
tar -hEdit: wtf... It's actually
tar -?. I'm so disappointedboom
tar eXtactZheVeckingFile
Me trying to decompress a .tar file
Joke's on you, .tar isn't compression
That's not going to stop me from getting confused every time I try!
You don't need the v, it just means verbose and lists the extracted files.
You don't need the z, it auto detects the compression
Yeah, I just tell our Linux newbies
tar xf, as in "extract file", and that seems to stick perfectly well.That's still kinda new. It didn't always do that.
Per https://www.gnu.org/software/tar/, it's been the case since 2004, so for about 19 and a half years...
Telling someone that they are Old with saying they are old…
Something something don't cite the old magics something something I was there when it was written...
Right, but you have no way of telling what version of tar that bomb is running
You may not, but I need it. Data anxiety is real.
tar -xzf
(read with German accent:) extract the files
Ixtrekt ze feils
German here and no shit - that is how I remember that since the first time someone made that comment
Same. Also German btw 😄
Not German but I remember the comment but not the right letters so I would have killed us all.
That’s so good I wish I needed to memorize the command
tar -uhhhmmmfuckfuckfuck
The Fish shell shows me just the past command with tar So I don't need to remember strange flags
I use zsh and love the fish autocomplete so I use this:
https://github.com/zsh-users/zsh-autosuggestions
Also have
fzfforctrl + rto fuzzy find previous commands.I believe it comes with oh-my-zsh, just has to be enabled in plugins and itjustworks™
man tar
you never said I can't run a command before it.
without looking, what’s the flag to push over ssh with compression
scp
not compressed by default
That's yet another great joke that GNU ruined.
Zip makes different tradeoffs. Its compression is basically the same as gz, but you wouldn't know it from the file sizes.
Tar archives everything together, then compresses. The advantage is that there are more patterns available across all the files, so it can be compressed a lot more.
Zip compresses individual files, then archives. The individual files aren't going to be compressed as much because they aren't handling patterns between files. The advantages are that an error early in the file won't propagate to all the other files after it, and you can read a file in the middle without decompressing everything before it.
Yeah that's a rather important point that's conveniently left out too often. I routinely extract individual files out of large archives. Pretty easy and quick with zip, painfully slow and inefficient with (most) tarballs.
Can you evaluate the directory tree of a tar without decompressing? Not sure if gzip/bzip2 preserve that.
Nowhere in here do you cover bzip, the subject of this meme. And tar does not compress.
It's just a different layer of compression. Better than gzip generally, but the tradeoffs are exactly the same.
Well, yes. But your original comment has inaccuracies due to those 2 points.
Obligatory shilling for unar, I love that little fucker so much
unar <yourfile>unzipdoesn'tgonna start lovingly referring to good software tools as “little fuckers”
Happy cake day!
cheers!
What weird Japanese zip files are you handling?
Voicebanks for Utau (free (as in beer, iirc) clone of Vocaloid) are primarily distributed as SHIFT-JIS encoded zips. For example, try downloading Yufu Sekka's voicebank: http://sekkayufu.web.fc2.com/ . If I try to
unzipthe "full set" zip, it produces a folder called РсЙ╠ГЖГtТPУ╞Й╣ГtГЛГZГbГgБi111025Бj. But unar detects the encoding and properly extracts it as 雪歌ユフ単独音フルセット（111025). I'm sure there's some flag you can pass tounzipto specify the encoding, but I like havingunarhandle it for me automatically.Ah, that's pretty cool. I'm not sure I know of that program. I do know a little vocaloid though, but I only really listen to 稲葉曇(Inabakumori).
I know inabakumori! Their music is so cool! When I first listened to rainy boots and lagtrain, it made me feel emotions I thought I had forgotten a long time ago... I wish my japanese was good enough to understand the lyrics without looking them up ._. I'm also a huge fan of Kikuo. His music is just something completely unique, not to mention his insane tuning. He makes Miku sing in ways I didn't think were possible lol
I get you, I want to learn more Japanese. I only understand a very small amount at this point. I don't have any Miku songs that I have really wanted to listen to, but that could change. I might check out Kikuo then. Also I love the animations Inabakumori release with their songs too. They have some new stuff that's really good if you haven't checked it out yet.
I'm an
atools kinda personLooks cool, I'll check it out
Where .7z at
On windows.
7z is available for Linux as well (CLI only)
It is open-source too.
I know, but I’d say ppl on Linux tend to not use it.
.7z gang, represent
When I was on windows I just used 7zip for everything. Multi core decompress is so much better than Microsoft's slow single core nonsense from the 90s.
Small
dickpackage kings/queens rise up.Yeah, 7z is the clear winner.
It's much slower to decompress than DEFLATE ZIP though
cool people use zst
(actually decreases decompression time by 1300%)
You can't decrease something by more than 100% without going negative. I'm assuming this doesn't actually decompress files before you tell it to.
Does this actually decompress in 1/13th the time?
Yeah, Facebook!
Sucks but yes that tool is damn awesome.
Meta also works with CentOS Stream at their Hyperscale variant.
Makes sense. There are actual programmers working at facebook. Programmers want good tools and functionality. They also just want to make good/cool/fun products. I mean, check out this interview with a programmer from pornhub. The poor dude still has to use jquery, but is passionate to make the best product they can, like everone in programming.
Right, I usually do that or lz4.
.tar.xz
xz is quite slow though
There's several levels you can use to trade off additional space for requiring more processing power. That being said, I hate xz and it still feels slow AF every time I use it.
It starting 0.5 seconds slower than usual saved us all a bit of a headache as it turns out.
I hate 7z, it's slow (at least for me) and for some reason I often have problems with these files
tar c file | pxz > file.tar.xz
pixz is in "extra" repo in arch. Same as pigz.
Same algo as in 7z
I had no idea about that!
Yeah, it's similar enough to tar.gz to always confuse me.
wait until you learn about
.tar.lzTar lzma nuts, amirite?
TAR LAZER!
Good for image backups, after zeroing empty space.
When I'm feeling cool and downloading a
*.tar*file, I'llwgetto stdout, and tar from stdin. Archive gets extracted on the fly.I have (successfully!) written an
.isoto CD this way, too (pipe wget to cdrecord). Fun stuff.Something like
wget avc.com | tar xvf?Almost, I think.
Didnt think this would ever work
This is what we call UNIX-way
TAr stands for Tape Archive. Tapes store data sequentially. Downloads are done sequentially.
It's really just like a far away tape drive.
I usually suppress output of either wget (-q) or of tar (no v flag), otherwise I think the output gets mangled and looks funny (you see both download progress and files being extracted).
...or shipping to MSDOS
Where's .7z people?
Can someone explain why MacOS always seems to create _MACOSX folders in zips that we Linux/Windows users always delete anyway?
Window adds desktop.ini randomly too
Linux adds .demon_portal files all over my computer too.
That's not Linux doing that. It's the demons in your hardware trying to escape. They normally don't cause too many issues luckily, but if you don't close the portals occasionally they can take over your system.
Yeah, those tend to be pre-folder settings for the File Explorer.
Like View options, thumbnails and such.
It's been a while for me, but I think there was something specially for thumbnails too. You might find one if you go into the folder options and set a folder to optimized for pictures/videos and add some to it.
Anti Commercial-AI license
You're thinking of the
thumbs.dbfiles: https://en.wikipedia.org/wiki/Windows_thumbnail_cache#Thumbs.dbHuh, never noticed that. Probably always thought that was just part of the program/files needed.
this is a complete uneducated guess from a relatively tech-illiterate guy, but could it contain mac-specific information about weird non-essential stuff like folder backgrounds and item placement on the no-grid view?
Correct. It contains filesystem metadata that's not supported in the zip files "filesystem".
Interesting. We have that in Linux too just not for every directory.
They're Metadata specific for Macs.
If you download a third party compression tool they'll probably have an option somewhere to exclude these from the zips but the default tool doesn't Afaik.
Thanks! Hmm, never thought of looking at 7zip's settings to see if it can autodelete/not unpack that stuff. I'll see if I can find such a setting!
You can definitely check, but I would expect the option to exist when the archive is created rather than when it's extracted
"Resource forks" IIRC, old stuff. Same for the .DS_Store file.
For just $12.99 you can disable this https://apps.apple.com/us/app/blueharvest/id739483376
Because Apple always gotta fuck with and "innovate" perfectly working shit
Windows's built-in tool can make zips without fucking with shit AND the resulting zip works just fine across systems.
Mac though...Mac produced zips always ALWAYS give me issues when trying to unzip on a non-mac (ESPECIALLY Linux)
MacOS has two files per file, so the extras need to be stored somewhere.
HFS+ has a different features set than NTFS or ext4, Apple elect to store metadata that way.
I would imagine modern FS like ZFS or btrfs could benefit from doing something similar but nobody has chosen to implement something like that in that way.
Yeah totally!
frantically searches for the meaning of all those abbreviations
I gotcha:
Great summary, but I've to add that NTFS is WAY more stable than ext4 when it comes to hardware glitches and/or power failures. ZFS is obviously superior to both but overkill for most people, BTRFS should be a nice middle ground and now even NAS manufacturers like Synology are migrating ext4 into BTRFS.
Well that's good to know because I had some terrible luck with it about a decade ago. Although I don't think I would go back to windows, I just don't need it for work anymore and it's become far too complex.
I've also had pretty bad luck with BTRFS though, although it seems to have improved a lot in the past 3 years that I've been using it.
ZFS would be good but having to rebuild the kernel module is a pain in the ass because when it fails to build you're unbootable (on root). I also don't like how clones are dependant on parents, requires a lot of forethought when you're trying to create a reproducible build on eg Gentoo.
Thanks!
.fitgirlrepack
I'm the weird one in the room. I've been using 7z for the last 10-15 years and now
.tar.zst, after finding out that ZStandard achieves higher compression than 7-Zip, even with 7-Zip in "best" mode, LZMA version 1, huge dictionary sizes and whatnot.zstd --ultra -M99000 -22 files.tar -o files.tar.zstYou can actually use Zstandard as your codec for 7z to get the benefits of better compression and a modern archive format! Downside is it's not a default codec so when someone else tries to open it they may be confused by it not working.
That is an interesting implementation of security through obscurity…
How does one enable this on the standard 7Zip client?
On Windows, it's easy! Unfortunately, on Linux, as far as I know, you currently have to use a non-standard client.
Cool, THANKS!! 😊 🙏
.tar.7z gang (probably not a good idea)
.tar.xz is. Or .tar.lzip.
.tar.7z.zip.bz2 gang unite
I use .tar.gz in personal backups because it's built in, and because its the command I could get custom subdirectory exclusion to work on.
Mf’ers act like they forgot about zstandatd
7z gang joined the chat.....
Me removing the plastic case of a 2.5' sata ssd to make it physically smaller
That's a big drive.
I mean xz/7z has kind of been the way for at least a decade now
Well, tar.zstd is starting to be the thing now.
First bundling everything in a tar file just to compress the thing in an individual step is kinda stupid, though. Everything takes much longer because of that. If you don't need to preserve POSIX permissions, tar is pointless anyway.
Slower, yes. More compression, yes. Stupid, no. tar serves a purpose beyond persevering permissions.
Use an archiving format that does both at once then, preserving whatever tar use cases has and compressing. The two steps are stupid, no arguing against that.
ZSTD FTW
Damn you guys borrowing consonants from polish names now?
Wut?
I think it's a joke about how people from Poland have a lot of consonant letters in their name, particularly the letter Z. This appears strange to non-poles, and thus became the subject of many jokes. So ZSTD looks a little bit like a Polish name in the sense that it's made up of many consonant letters, including a Z.
More examples of people making fun of Polish names:
Here is an alternative Piped link(s):
https://www.piped.video/watch?v=AfKZclMWS1U
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I'm open-source; check me out at GitHub.
Thank you PipedLinkBot
This sent me down a Polish rabbit hole I'm very happy to have fell into
.tar.gz
Can we please just never use proprietary rar ever. We have 7z, tar.gz, and the classic zip
all the cool kids use .cab
I use the command line every day, but can't be bothered with all the compression options of tar and company.
zip -r thing.zip things/andunzip thing.zipare temptingly more straightforward.Need more compression?
zip -r -9 thing.zip things/. Need a faster option? Use a smaller digit."yes i would love to
tar -xvjpfmy files"-- statement dreamed up by the utterly insane
Present, I'm the
tar cvJfinsaneZip is fine (I prefer 7z), until you want to preserve attributes like ownership and read/write/execute rights.
Some zip programs support saving unix attributes, other - do not. So when you download a zip file from the internet - it's always a gamble.
Tar + gzip/bz2/xz is more Linux-friendly in that regard.
Also, zip compresses each file separately and then collects all of them in one archive.
Tar collects all the files first, then you compress the tarball into an archive, which is more efficient and produces smaller size.
yes, and you still need zhe mnemonics
There’s gotta be a buncha tools that Clippy into the terminal to say “did you mean ____?” right? Including some new ones where they trained/fine-tuned a language model on man pages?
Interesting it’s not the most popular thing to use a GUI and use shortcuts for everything you want to do while still having the option to click through a menu or wizard for whatever you haven’t memorized. I suppose the power and speed of the command line are difficult to match if you introduce anything else, and if you spend time using a user interface that’s time you can’t spend honing your command line skills.
There's
thefuck, but it hasn't given me good suggestions.tar xf things.tar.gz/bz2/etc Should be enough to extract. It can usually figure out the compression automatically.
The problem with that is that it will not preserve flags and access rights.
how about
.tar.zipor.tar.rar?I prefer .tar.rar.tar.
.tar.zip.tar.xzall the wayZip, RAR, 7z etc. store and compress the files. Tarballs work differently, tar stores the files and the second program compresses the tar as a continuous stream.
They originated in different mediums, programs like zip were born to deal with folder structures, tar was created to deal with linear tape archives (hence the name).
TIL
Kind of redundant. Both
.zipand.rarstore an index of files within the archive and are a bit 'inside-out' when it comes what we get fromtar.gz.That is, ZIP is pretty close to what you'd get if you first gzipped all your files and then put them into a
.tar.RAR does a little more (if I remember correctly), such as generating a dictionary of common redundancies between files and then uses that knowledge to compress the files individually, but better. Something akin to a
.tarfile is still the result though.That's fine. I'm use to being unrepresented in the arj, lha, and uc2 crew
If you download and extract the tarball as two separate steps instead of piping
curldirectly intotar xz(for gzip) /tar xj(for bz2) /tar xJ(for xz), are you even a Linux user?I download and then tar. Curl pipes are scary
They really, really aren’t. Let’s take a look at this command together:
Sorry the formatting's a bit messy, Lemmy's not having a good day today
This command will start to extract the tar file while it is being downloaded, saving both time (since you don’t have to wait for the entire file to finish downloading before you start the extraction) and disk space (since you don’t have to store the .tar file on disk, even temporarily).
Let’s break down what these scary-looking command line flags do. They aren’t so scary once you get used to them, though. We’re not scared of the command line. What are we, Windows users?
You may have noticed also that in the first command I showed, I didn’t put a - in front of the arguments to tar. This is because the tar command is so old that it takes its arguments BSD style, and will interpret its first argument as a set of flags regardless of whether there’s a dash in front of them or not. tar -xz and tar xz are exactly equivalent. tar does not care.
Thanks for the explanation, I might use more pipes now. Is it correct, that tar will restore the files in the tarball in the current directory?
Yes. You can specify
tar -C somedirif you want it to extract them somewhere else.As a rule of thumb, I always extract my tarballs in a newly created, empty directory, just in case whoever packed it didn't put all its files in a subdir
It's not scary from the flags, but rather what is inside the tar/zip.
the problem is if the connection gets interrupted your progress is gone. you download to a file first and it gets interrupted, you just resume the download later
.tar.xz
☠️
one of the cool things about tar is that it's hard link aware
Good ol -xvjpf
tar cvjf compressed-shit.tar.bz2 /path/to/uncompressed/shit/
Only way to fly.
I stopped doing that because I found it painfully slow. And it was quicker to gzip and upload than to bzip2 and upload.
Of course, my hardware wasn't quite as good back then. I also learned to stop adding 'v' flag because the bottleneck was actually stdout! (At least when extracting).
for the last 14 years of my career, I was using stupidly overpowered Oracle exadata systems exclusively, so "slow" meant 3 seconds instead of 1.
Now that I'm retired, I pretty much never need to compress anything.
I’m curious about the contents in your compressed shit.
.txz, I'm too lazy to type the full nameI'm from
tar.gz.gpggang to keep away nasty storage providersThis guy tar balls
What's wrong with .7z?
Nothing, but I've read people who act as if tar files are some sort of alien artifact ready to rip their faces off.
.tar.zst forever
Bzip2 compression is often surprisingly good with text files, especially log files. It seems to "see" redundancies there - and logs often have a lot of it - far better than gzip and sometimes even lzma.
Anyway, if I saw a bunch of
tar.bz2files, that's what I'd expect to find in them.i just use either zip or tar+gzip. personally, i avoid rar as it's a proprietary format.
most of the things i want to send around my network in archives are already compressed binary files, so i just tar everything.
You reinvented zip and didn't even know it.
tarwas nearly and adult whenzipwas born.See my reply here: https://midwest.social/comment/10257041
The two aren't really equivalent. They make different tradeoffs. The scheme of "compress individual files, then archive" from GP is what zip does. Tar does "archive first, then compress the whole thing".
The link does not load for some reason, but tar itself does not compress anything. Compression can (and usually is) applied afterwards, but that's an additional integration that is not part of Tape ARchive, as such.
Yes, I'm aware of how this works.
The important point is that zip and compressed tarballs have overlapping but not identical purposes.
.zlibI unironically used xz for a long time. It was just eazy and all around very good compress. A close second is 7zip because I used it on windows for years.
so silly loll
I still wonder what that's like. Somebody must still occasionally get a notification that SOMEWHERE somebody paid for their WinRAR license and is like "WOAH WE GOT ANOTHER ONE!"
Never looked back since 7z though. :D
Make the stage 4 tarball
I’m in this picture and I resent it.
Compatibility aside, I'd say that
.tar.pxzaka.tpxzis probably my vote.LZMA is probably what I'd want to use.
xzand7zipuse that. It's a bit slow to compress, but it has good compression ratios, and it's faster to decompress than bzip2.pixzpermits for parallel LZMA compression/decompression. On present-day processors with a lot of cores, that's desirable.https://github.com/vasi/pixz
It also can use .tar as its container format, which is desirable; that's everywhere.
The major drawback to .tar is that it doesn't support indexed access, so extracting a single file isn't fast, but .tar.pxz does.
Im a pirate, I am rar like, roawr, lol
What the fuck is that file extension?