2026-04-01 09:32:52 iv4nshm4k0v: Yeah I'm talking about using the old compression formats, not new ones 2026-04-01 09:40:52 I kinda read that differently. "08:47 You can make a .zip with the latest compression algorithms and it will still open with the oldest and crappiest .zip extraction programs, because the format's still the same" - which, I believe, is not /quite/ what would happen. 2026-04-01 09:40:52 If Wikipedia is to be believed, Bzip2 was implemented in PKZIP 4.6 (2001), but first mentioned in APPNOTE 5.2 (2003.) So I presume that PKUNZIP 2.0 for DOS from 1993 would be able to list archive members compressed with Bzip2, but not actually extract them. 2026-04-01 09:43:10 (Nor would such .zip archives conform to the ISO standard, so interoperability may be limited even with new extraction software.) 2026-04-01 10:31:11 My point is more that modern compression algorithms do a better job with DEFLATE, and yet can still be opened by ancient .zip software 2026-04-01 10:31:54 Not "you can use zstd and open it with Windows 98" 2026-04-01 10:34:46 If you look at the Info-ZIP manpage you'll see they recommend against using bzip2 unless you know the corresponding unzip that will be used supports it, it's not the default 2026-04-01 10:35:02 Any specific software to look into for better deflate compression? 2026-04-01 10:35:31 libz is better than it was in the 90's for one 2026-04-01 10:36:47 I found this out when I was working on an old self-extracting bootloader, it turned out to make an updated image 'fit' without rebuilding it was quite easy as DEFLATE is self-terminating, and the latest version of deflate would consistently beat the old compressed size 2026-04-01 10:37:12 So I could just overlay a new image and it would work 2026-04-01 10:38:05 Also a lot of compression streams are unchecksummed because they are extremely likely to fail to inflate if there are any bit errors at all 2026-04-01 10:38:19 libz / zlib 2026-04-01 10:40:13 IIRC, .xz applies CRC-64 to the /compressed/ stream - which proved to be a controversial design decision. 2026-04-01 10:43:09 Why is it controversial? 2026-04-01 10:47:03 On a package manager I worked on we considered .xz and realised .gz was way faster including both downloading over a slow connection and decompressing 2026-04-01 10:47:31 So I'm not really a fan of .xz anyway but I guess if your upload quota is the issue then it's the right choice 2026-04-01 10:52:02 It's nice that libera.chat want IRC to flourish but this just sounds dumb https://libera.chat/news/downgrades 2026-04-01 10:53:13 > our recent feature rollout has been hampered by a major flaw in IRC: client diversity. Clients are free to support or not support any set of IRCv3 features, which means that not everyone using Libera.Chat gets the same experience 2026-04-01 10:53:50 What the hell are they talking about, maybe not everyone wants 'typing' notifications 2026-04-01 10:55:34 Oh I think I may have fallen for a date related news article 2026-04-01 10:56:09 Okay the problem is I have so little interaction with libera.chat staff that I could seriously believe they'd say something that stupid sincerely 2026-04-01 10:56:25 ACTION is the fool today 2026-04-01 10:59:29 I mean everyone's a loser when your april fools is to say something really dumb, and everyone believes it was sincere because they have a low opinion of you 2026-04-01 14:12:34 veltas: I mean the criticism at http://nongnu.org/lzip/xz_inadequate.html , particularly regarding such CRCs being potential sources of false error detection: either when CRC itself becomes damaged (while the compressed stream is intact), or when the bitstream is damaged in such a way that does not affect decompression in any way. Can't say I quite understand it, but the author cites sources. 2026-04-01 14:52:10 new https://labynet.fr/infra.svg 2026-04-01 15:17:57 iv4nshm4k0v: Looks like xz the container of the LZMA stream is bad, but not really a comment on the compression format of LZMA itself 2026-04-01 15:18:46 The xz container seems like it was overengineered and made by people without experience 2026-04-01 15:19:13 If they had at least kept it simple it might have been better and avoided some of the issues 2026-04-01 15:30:44 Reading the spec https://tukaani.org/xz/xz-file-format-1.2.1.txt 2026-04-01 15:31:20 So it's a series of 'stream's and each stream starts with a 12 byte stream header 2026-04-01 15:31:34 And this 12 byte header stores 4 bits of actual information, apart from the magic number and checksum 2026-04-01 15:33:38 The stream footer has 2 magic bytes... that is er interesting 2026-04-01 15:34:00 To detect a truncated file, but if the stream is of a serious length then those 2 bytes have a decent chance of appearing 2026-04-01 15:39:11 iv4nshm4k0v: The more I read this spec the more I understand why people complained 2026-04-01 15:42:38 I'll have to read up on LZMA as that might be interesting and not just frustrating 2026-04-01 15:43:45 You might be better equipped to understand the issues than me, at any rate. (Remember that I've majored in physics, not CS.) Personally, I've ended up switching to .lz (or, at times, .zstd) where I've previously used .xz. Or .gz / .bz2 / .zip when interoperability demands that. Though people at #netbsd noted that lzip(1) is slowed on decompression than xz(1), somehow. 2026-04-01 15:44:14 In what scenario would you read xz backwards? I guess on tape, I wonder if that will impact anyone 2026-04-01 15:44:27 It's designed so you can read it backwards too 2026-04-01 15:44:46 Maybe that's to access the index so you can output the index sequentially while generating the stream 2026-04-01 15:45:05 Hey iv4nshm4k0v some of the smartest programmers I've known were physicists