Tar did a weird thing today
I'm so baffled I had to ask – why this behaviour?
cd /var/www/html
tar czf ~/package.tgz admin/* api/* mobile/*
I do this, and the resulting package doesn't include a couple of hidden files – api/.htaccess and admin/.htaccess. However...
cd /var/www/html
tar czf ~/package.tgz *
This time the hidden .htaccess files are there.
Does anybody have enlightenment to offer as to why?
*in your commands is expanded by the shell before tar sees them. It also does not expand hidden files.So when you do
admin/*the shell expands to all non hidden files inside admin. Which does not includeadmin/.htaccess. So tar is never told to archive this file, only the other non hidden files and folders. It will still archive hidden files and folders nested deeper though.In the second example
*expands toadminand the other does which are not hidden at that level. Then tar can open these dirs and recursivly archive all files and folders including the hidden ones.You can see what commands actually get executed after any shell expansions if you run
set -xfirst. Thenset +xto turn that off again.Here is an example using
ls:A quicker way to test this is by using echo
try
echo tar czf ~/package.tgz admin/* api/* mobile/*You don't need the wildcard, and as others have pointed out, it doesn't include "hidden " dot files by default.
tar -czf ~/package.tgz admin api mobileAh yes, one of the biggest bugs-later-turned-design-mistakes of UNIX. This is not tar, this is your shell; you always need to use both
*and.*for a wildcard to match all files in directory - e.g.tar czf ~/package.tgz admin/* admin/.* api/* api/.* mobile/* mobile/.*Thanks - this is what I did with a 'you had one job' look beaming at the terminal after realising the hidden files were missed and indeed it did the trick.
Could this be made easier by setting globstar or dotglob options, to include hidden files but not try to grab . and .. Directory files
Thank you all! Over 25 years on Linux and still new learnings to discover...
Probably due to wildcard expansion by the shell. Use dot (.) instead of asterisk (*).
This is potentially a great 'weeder' question for junior Unix admin interviews, as it requires some knowledge about shell globbing and tar dir traversal.
I admit it took me a sec (and a second read) before I got it, so it was a fun "hey what" exercise.
Excellent question.
Dang as soon as you said globbing I realized what had happened but didn't see it right away either
https://unix.stackexchange.com/questions/40662/what-is-the-setting-in-bash-for-globbing-to-control-whether-matches-dot-files#40663
Sure. It's important to first realize that the * character is not read by tar, but by your shell which first expands it into a list of files/directories that are then passed into tar to process. In the first command you are asking your shell to expand all (implicitly non-hidden) files in the listed directories. In the second you are doing the same, but one level up.
What happens then is that when tar is given a directory as an input to a create command, it will include everything in that directory hidden files included. This is why the second command worked as you expected. In the first command you are providing tar with an explicit list of files which does not include those which are hidden due to how shell glob expansion works.
You can do a
set -xbefore running these commands and this will show you the exact command line after shell expansion before execution. That should give a clearer view of why you see different behavior.