• @sik0fewl@lemmy.ca
      link
      fedilink
      011 months ago

      I stopped doing that because I found it painfully slow. And it was quicker to gzip and upload than to bzip2 and upload.

      Of course, my hardware wasn’t quite as good back then. I also learned to stop adding ‘v’ flag because the bottleneck was actually stdout! (At least when extracting).

      • @robolemmy@lemmy.world
        link
        fedilink
        English
        011 months ago

        for the last 14 years of my career, I was using stupidly overpowered Oracle exadata systems exclusively, so “slow” meant 3 seconds instead of 1.

        Now that I’m retired, I pretty much never need to compress anything.

  • @AVincentInSpace@pawb.social
    link
    fedilink
    English
    011 months ago

    If you download and extract the tarball as two separate steps instead of piping curl directly into tar xz (for gzip) / tar xj (for bz2) / tar xJ (for xz), are you even a Linux user?

    • @brian@programming.dev
      link
      fedilink
      011 months ago

      the problem is if the connection gets interrupted your progress is gone. you download to a file first and it gets interrupted, you just resume the download later

      • @AVincentInSpace@pawb.social
        link
        fedilink
        English
        0
        edit-2
        11 months ago

        They really, really aren’t. Let’s take a look at this command together:

        curl -L [some url goes here] | tar -xz
        

        Sorry the formatting’s a bit messy, Lemmy’s not having a good day today

        This command will start to extract the tar file while it is being downloaded, saving both time (since you don’t have to wait for the entire file to finish downloading before you start the extraction) and disk space (since you don’t have to store the .tar file on disk, even temporarily).

        Let’s break down what these scary-looking command line flags do. They aren’t so scary once you get used to them, though. We’re not scared of the command line. What are we, Windows users?

        • curl -L – tells curl to follow 3XX redirects (which it does not do by default – if the URL you paste into cURL is a URL that redirects (GitHub release links famously do), and you don’t specify -L, it’ll spit out the HTML of the redirect page, which browsers never normally show)
        • tar -x – eXtract the tar file (other tar “command” flags, of which you must specify exactly one, include -c for Creating a tar file, and -t for Testing a tar file (i.e. listing all of the filenames in it and making sure their checksums are okay))
        • tar -z – tells tar that its input is gzip compressed (the default is not compressed at all, which with tar is an option) – you can also use -j for bzip2 and -J for xz
        • tar -f which you may be familiar with but which we don’t use here – -f tells tar which file you want it to read from (or write to, if you’re creating a file). tar -xf somefile.tar will extract from somefile.tar. If you don’t specify -f at all, as we do here, tar will default to reading the file from stdin (or writing a tar file to stdout if you told it to create). tar -xf somefile.tar (or tar -xzf somefile.tar.gz if your file is gzipped) is exactly equivalent to cat somefile.tar.gz | tar -xz (or tar -xz < somefile.tar – why use cat to do something your shell has built-in?)
        • tar -v which you may be familiar with but which we don’t use here – tells tar to print each filename as it extracts the file. If you want to do this, you can, but I’d recommend telling curl to shut up so it doesn’t mess up the terminal trying to show download progress also: curl -L --silent [your URL] | tar -xvz (or -xzv, tar doesn’t care about the order)

        You may have noticed also that in the first command I showed, I didn’t put a - in front of the arguments to tar. This is because the tar command is so old that it takes its arguments BSD style, and will interpret its first argument as a set of flags regardless of whether there’s a dash in front of them or not. tar -xz and tar xz are exactly equivalent. tar does not care.

  • @frezik@midwest.social
    link
    fedilink
    011 months ago

    Zip makes different tradeoffs. Its compression is basically the same as gz, but you wouldn’t know it from the file sizes.

    Tar archives everything together, then compresses. The advantage is that there are more patterns available across all the files, so it can be compressed a lot more.

    Zip compresses individual files, then archives. The individual files aren’t going to be compressed as much because they aren’t handling patterns between files. The advantages are that an error early in the file won’t propagate to all the other files after it, and you can read a file in the middle without decompressing everything before it.

    • @herrvogel@lemmy.world
      link
      fedilink
      011 months ago

      Yeah that’s a rather important point that’s conveniently left out too often. I routinely extract individual files out of large archives. Pretty easy and quick with zip, painfully slow and inefficient with (most) tarballs.

    • fmstrat
      link
      fedilink
      0
      edit-2
      11 months ago

      Nowhere in here do you cover bzip, the subject of this meme. And tar does not compress.

      • @frezik@midwest.social
        link
        fedilink
        011 months ago

        It’s just a different layer of compression. Better than gzip generally, but the tradeoffs are exactly the same.

        • fmstrat
          link
          fedilink
          011 months ago

          Well, yes. But your original comment has inaccuracies due to those 2 points.

  • Eager Eagle
    link
    fedilink
    English
    0
    edit-2
    11 months ago

    I use the command line every day, but can’t be bothered with all the compression options of tar and company.

    zip -r thing.zip things/ and unzip thing.zip are temptingly more straightforward.

    Need more compression? zip -r -9 thing.zip things/. Need a faster option? Use a smaller digit.

    • @lseif@sopuli.xyz
      link
      fedilink
      011 months ago

      “yes i would love to tar -xvjpf my files”

      – statement dreamed up by the utterly insane

        • @brbposting@sh.itjust.works
          link
          fedilink
          011 months ago

          There’s gotta be a buncha tools that Clippy into the terminal to say “did you mean ____?” right? Including some new ones where they trained/fine-tuned a language model on man pages?

          Interesting it’s not the most popular thing to use a GUI and use shortcuts for everything you want to do while still having the option to click through a menu or wizard for whatever you haven’t memorized. I suppose the power and speed of the command line are difficult to match if you introduce anything else, and if you spend time using a user interface that’s time you can’t spend honing your command line skills.

    • janAkali
      link
      fedilink
      English
      0
      edit-2
      11 months ago

      Zip is fine (I prefer 7z), until you want to preserve attributes like ownership and read/write/execute rights.

      Some zip programs support saving unix attributes, other - do not. So when you download a zip file from the internet - it’s always a gamble.
      Tar + gzip/bz2/xz is more Linux-friendly in that regard.

      Also, zip compresses each file separately and then collects all of them in one archive.
      Tar collects all the files first, then you compress the tarball into an archive, which is more efficient and produces smaller size.

  • @MonkeMischief@lemmy.today
    link
    fedilink
    011 months ago

    I still wonder what that’s like. Somebody must still occasionally get a notification that SOMEWHERE somebody paid for their WinRAR license and is like “WOAH WE GOT ANOTHER ONE!”

    Never looked back since 7z though. :D

      • @woelkchen@lemmy.world
        link
        fedilink
        011 months ago

        First bundling everything in a tar file just to compress the thing in an individual step is kinda stupid, though. Everything takes much longer because of that. If you don’t need to preserve POSIX permissions, tar is pointless anyway.

          • @woelkchen@lemmy.world
            link
            fedilink
            011 months ago

            Use an archiving format that does both at once then, preserving whatever tar use cases has and compressing. The two steps are stupid, no arguing against that.

  • @jh29a@lemmy.blahaj.zone
    link
    fedilink
    0
    edit-2
    11 months ago

    I use .tar.gz in personal backups because it’s built in, and because its the command I could get custom subdirectory exclusion to work on.