Spyke

Syndicated from the fediverse. Read and engage on the original instance.

View original on lemmy.ml

TIL XZ has built-in checksumming (and now threaded decompression?)

I need to transport multiple very large files over an unstable and untrusted network, and the file contents are outputted as a data stream. I wanted to use OpenSSL for streaming authenticated encryption, but they purposefully don't support that and are preachy about it.

Well, it turns out that XZ has checksumming built-in! It even has different algorithms (CRC32, CRC64, and SHA256). It's part of the same file, within/before the encryption, and automatically verified by the decompression tool. I'm already using XZ for compression before encryption, so this is just super convenient and useful. Also, it seems like XZ supports threaded decompression now, when it didn't before. Thanks XZ devs!

View original on lemmy.ml
35

2 replies

lemmy.world

rsync will also verify the contents of files it copies, as will syncthing.

4

You reached the end

TIL XZ has built-in checksumming (and now threaded decompression?) | Spyke