R.L. Dane :Debian: :OpenBSD: 🍵 :MiraLovesYou:<p>New <a href="https://polymaths.social/tags/blog" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>blog</span></a> post: Unix Data Compression Shootout</p><p><a href="https://rldane.space/unix-data-compression-shootout.html" rel="nofollow noopener noreferrer" target="_blank">https://rldane.space/unix-data-compression-shootout.html</a></p><p>881 words</p><p>cc: my wonderful <a href="https://polymaths.social/tags/chorus" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>chorus</span></a>: <span class="h-card"><a href="https://polymaths.social/@joel" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>joel</span></a></span> <span class="h-card"><a href="https://polymaths.social/@dm" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>dm</span></a></span> <span class="h-card"><a href="https://polymaths.social/@sotolf" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>sotolf</span></a></span> <span class="h-card"><a href="https://polymaths.social/@thedoctor" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>thedoctor</span></a></span> <span class="h-card"><a href="https://merveilles.town/@pixx" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>pixx</span></a></span> <span class="h-card"><a href="https://behold.mylegendary.quest/users/twizzay" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>twizzay</span></a></span> <span class="h-card"><a href="https://polymaths.social/@orbitalmartian" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>orbitalmartian</span></a></span> <span class="h-card"><a href="https://fosstodon.org/@adamsdesk" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>adamsdesk</span></a></span> <span class="h-card"><a href="https://polymaths.social/@krafter" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>krafter</span></a></span> <span class="h-card"><a href="https://polymaths.social/@roguefoam" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>roguefoam</span></a></span> <span class="h-card"><a href="https://polymaths.social/@clayton" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>clayton</span></a></span> <span class="h-card"><a href="https://fosstodon.org/@giantspacesquid" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>giantspacesquid</span></a></span></p><p>(I will happily add/remove you from the chorus upon request! :)</p><p><a href="https://polymaths.social/tags/100daystooffload" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>100DaysToOffload</span></a> #32</p><p><a href="https://polymaths.social/tags/rldanewriting" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>rlDaneWriting</span></a> <a href="https://polymaths.social/tags/unix" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Unix</span></a> <a href="https://polymaths.social/tags/linux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Linux</span></a> <a href="https://polymaths.social/tags/bsd" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>BSD</span></a> <a href="https://polymaths.social/tags/datacompression" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataCompression</span></a> <a href="https://polymaths.social/tags/gzip" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>gzip</span></a> <a href="https://polymaths.social/tags/bzip" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>bzip</span></a> <a href="https://polymaths.social/tags/bzip2" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>bzip2</span></a> <a href="https://polymaths.social/tags/bzip3" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>bzip3</span></a> <a href="https://polymaths.social/tags/zstd" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>zstd</span></a> <a href="https://polymaths.social/tags/zst" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>zst</span></a> <a href="https://polymaths.social/tags/xz" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>xz</span></a> <a href="https://polymaths.social/tags/lz4" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>lz4</span></a> <a href="https://polymaths.social/tags/compression" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>compression</span></a></p>
Recent searches
No recent searches
Search options
Only available when logged in.
shakedown.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
A community for live music fans with roots in the jam scene. Shakedown Social is run by a team of volunteers (led by @clifff and @sethadam1) and funded by donations.
Administered by:
Server stats:
246active users
shakedown.social: About · Status · Profiles directory · Privacy policy
Mastodon: About · Get the app · Keyboard shortcuts · View source code · v4.3.7
#datacompression
0 posts · 0 participants · 0 posts today
Farooq | فاروق<p>So <span class="h-card" translate="no"><a href="https://polymaths.social/@rl_dane" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>rl_dane</span></a></span> introduced <a href="https://cr8r.gg/tags/bzip3" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>bzip3</span></a> to me to use instead of <a href="https://cr8r.gg/tags/bzip2" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>bzip2</span></a>. Let's turn some <code>bz2</code> files into <code>bz3</code> to see the difference.</p><p><strong>First example: 90k opus files</strong></p><p>hey snips wake word dataset. It has ~90k opus files and a tar file of <code>3.1GB</code>. <code>bzip2</code> produces the same <code>3.1GB</code> which is as expected. bzip3 created <code>3.0GB</code> but used tons of computation power. Not worth the <code>100MB</code></p><p><strong>Second example: Windows 7 virtual box VM image</strong></p><p><code>Windows7.vdi</code> it's Windows 7 VM image for the "special" days. I think I have to get rid of it. But while it is still there, let's see how each will perform. It is <code>16GB</code> uncompressed. <code>bzip2 -9</code> is <code>7.0GB</code>. <code>bzip3</code> is <code>6.3GB</code> but at the expense of like 3x CPU time. Deleting all of them anyway. Down with Windows.</p><p><strong>Third example: Pure XML text file</strong></p><p>Pure XML file. It's Persian and English characters. Uncompressed is <code>1.7GB</code>. <code>bzip2 -9</code> is <code>276M</code> while <code>bzip3</code> is <code>260MB</code></p><p><strong>Final example: Creating a simple bomb</strong></p><p>So I did this:</p><pre><code>dd if=/dev/zero of=./justzero bs=2G count=6<br></code></pre><p>So now I have a <code>16GB</code> with only zero bytes. <code>bzip2 -9</code> is <code>672KB</code>. <code>bzip3</code> is <code>46KB</code>. </p><p><strong>Conclusion</strong></p><p>Thank you <span class="h-card" translate="no"><a href="https://polymaths.social/@rl_dane" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>rl_dane</span></a></span> </p><p>Real nice thing!</p><p><a href="https://cr8r.gg/tags/compression" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>compression</span></a> <a href="https://cr8r.gg/tags/gzip" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>gzip</span></a> <a href="https://cr8r.gg/tags/zip" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>zip</span></a> <a href="https://cr8r.gg/tags/filecompression" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>filecompression</span></a> <a href="https://cr8r.gg/tags/textcompression" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>textcompression</span></a> <a href="https://cr8r.gg/tags/datacompression" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>datacompression</span></a> <a href="https://cr8r.gg/tags/linux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>linux</span></a> <a href="https://cr8r.gg/tags/unix" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>unix</span></a> <a href="https://cr8r.gg/tags/tech" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>tech</span></a></p>
Tweede golf<p>Our zlib-rs project implements a memory-safe and performant drop-in replacement for zlib, a widely-used data compression library.</p><p><span class="h-card" translate="no"><a href="https://hachyderm.io/@folkertdev" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>folkertdev</span></a></span> shares the status quo of zlib-rs, including the good news that performance for the highest compression level is on par with the zlib-ng fork of zlib.</p><p>Read the blog for all the details:</p><p><a href="https://tweedegolf.nl/en/blog/134/current-zlib-rs-performance" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">tweedegolf.nl/en/blog/134/curr</span><span class="invisible">ent-zlib-rs-performance</span></a></p><p><span class="h-card" translate="no"><a href="https://fosstodon.org/@trifectatech" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>trifectatech</span></a></span> </p><p><a href="https://fosstodon.org/tags/rustlang" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>rustlang</span></a> <a href="https://fosstodon.org/tags/datacompression" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>datacompression</span></a> <a href="https://fosstodon.org/tags/opensource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>opensource</span></a></p>
Norobiik @Norobiik@noc.social<p>Red Hat in all caps says STOP USAGE OF ANY FEDORA RAWHIDE INSTANCES</p><p><a href="https://noc.social/tags/RedHat" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>RedHat</span></a> on Friday warned that a malicious <a href="https://noc.social/tags/Backdoor" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Backdoor</span></a> found in the widely used <a href="https://noc.social/tags/DataCompression" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataCompression</span></a> library xz may be present in the upcoming <a href="https://noc.social/tags/FedoraLinux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>FedoraLinux</span></a> 40 and in the <a href="https://noc.social/tags/FedoraRawhide" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>FedoraRawhide</span></a> developer distribution.</p><p>Malicious <a href="https://noc.social/tags/SSH" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>SSH</span></a> backdoor sneaks into xz, <a href="https://noc.social/tags/Linux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Linux</span></a> world's data compression library<br><a href="https://go.theregister.com/feed/www.theregister.com/2024/03/29/malicious_backdoor_xz/" rel="nofollow noopener noreferrer" target="_blank"><span class="invisible">https://</span><span class="ellipsis">go.theregister.com/feed/www.th</span><span class="invisible">eregister.com/2024/03/29/malicious_backdoor_xz/</span></a></p>
R. L. Dane :debian: :openbsd:<p><span class="h-card"><a href="https://digipres.club/@misty" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>misty</span></a></span></p><p>Probably some form of Lempel-Ziv, but there are so many variants, to say nothing of implementations. </p><p>The forensics tool FOREMOST might be able to extract something, but it's highly unlikely. </p><p>Best bet would be to find a (lossless) data compression forum or subreddit and ask for help with it (providing sample files) or undergo the slow process of learning how various compressors work on a low level. </p><p>...</p><p><a href="https://fosstodon.org/tags/DataCompression" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DataCompression</span></a> <a href="https://fosstodon.org/tags/AskFedi" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>AskFedi</span></a></p>
ExploreLive feeds
Mastodon is the best way to keep up with what's happening.
Follow anyone across the fediverse and see it all in chronological order. No algorithms, ads, or clickbait in sight.
Create accountLoginDrag & drop to upload