Spyke

Tar did a weird thing today

I'm so baffled I had to ask – why this behaviour?

cd /var/www/html
tar czf ~/package.tgz admin/* api/* mobile/*

I do this, and the resulting package doesn't include a couple of hidden files – api/.htaccess and admin/.htaccess. However...

cd /var/www/html
tar czf ~/package.tgz *

This time the hidden .htaccess files are there.

Does anybody have enlightenment to offer as to why?

View original on sopuli.xyz
programming.dev

* in your commands is expanded by the shell before tar sees them. It also does not expand hidden files.

So when you do admin/* the shell expands to all non hidden files inside admin. Which does not include admin/.htaccess. So tar is never told to archive this file, only the other non hidden files and folders. It will still archive hidden files and folders nested deeper though.

In the second example * expands to admin and the other does which are not hidden at that level. Then tar can open these dirs and recursivly archive all files and folders including the hidden ones.

You can see what commands actually get executed after any shell expansions if you run set -x first. Then set +x to turn that off again.

Here is an example using ls:

$ set -x; ls -A foo/*; ls -A *; set +x
+ ls --color=tty -A foo/baz
foo/baz
+ ls --color=tty -A foo
.bar  baz
+ set +x
28

A quicker way to test this is by using echo

try echo tar czf ~/package.tgz admin/* api/* mobile/*

4

You don't need the wildcard, and as others have pointed out, it doesn't include "hidden " dot files by default.

tar -czf ~/package.tgz admin api mobile

18
lemmy.ml

Ah yes, one of the biggest bugs-later-turned-design-mistakes of UNIX. This is not tar, this is your shell; you always need to use both * and .* for a wildcard to match all files in directory - e.g. tar czf ~/package.tgz admin/* admin/.* api/* api/.* mobile/* mobile/.*

9

Thanks - this is what I did with a 'you had one job' look beaming at the terminal after realising the hidden files were missed and indeed it did the trick.

4

Could this be made easier by setting globstar or dotglob options, to include hidden files but not try to grab . and .. Directory files

1

Probably due to wildcard expansion by the shell. Use dot (.) instead of asterisk (*).

5
lemmy.ca

This is potentially a great 'weeder' question for junior Unix admin interviews, as it requires some knowledge about shell globbing and tar dir traversal.

I admit it took me a sec (and a second read) before I got it, so it was a fun "hey what" exercise.

Excellent question.

5

Dang as soon as you said globbing I realized what had happened but didn't see it right away either

1

Sure. It's important to first realize that the * character is not read by tar, but by your shell which first expands it into a list of files/directories that are then passed into tar to process. In the first command you are asking your shell to expand all (implicitly non-hidden) files in the listed directories. In the second you are doing the same, but one level up.

What happens then is that when tar is given a directory as an input to a create command, it will include everything in that directory hidden files included. This is why the second command worked as you expected. In the first command you are providing tar with an explicit list of files which does not include those which are hidden due to how shell glob expansion works.

You can do a set -x before running these commands and this will show you the exact command line after shell expansion before execution. That should give a clearer view of why you see different behavior.

1

You reached the end

Tar did a weird thing today | Spyke