shell | - 123Unix!com

clamscan exclude & include REGEX format

clamscan documentation is silent about which format the application expects for the –exclude=REGEX, –exclude-dir=REGEX, –include=REGEX, –include-dir=REGEX command line options.

Here are the REGEX rules found experimentally:

. is any symbol
* means zero or more occurrences of the preceding character
+ means 1 or more occurrences of the preceding character
^ ancors to the start of the filename or directory name with full path, either absolute or relative to CWD
/ is treated literally as a path subdirectories separator
leading / ancors to the beginning of the filename or directory name
trailing / ancors to the end of the directory name
\s \w \S and other escapes seem to have the backslash ignored
buy this point I got bored 🙂

What follows is the listing of the experiments shell session.

$ pwd
/tmp
$ mkdir d
$ mkdir d/e
$ touch d/f d/e/g
$ clamscan d
d/f: Empty file

$ clamscan --exclude-dir=f d
d/f: Empty file

$ clamscan -r --exclude-dir=f d
d/e/g: Empty file
d/f: Empty file

$ clamscan -r --exclude-dir=e d
d/e: Excluded
d/f: Empty file

$ clamscan -r --exclude-dir=/e d
d/e: Excluded
d/f: Empty file

$ mkdir d/e2 d/3e3 
$ clamscan -r --exclude-dir=/e d
d/e: Excluded
d/e2: Excluded
d/f: Empty file

$ clamscan -r --exclude-dir=/e/ d
d/e: Excluded
d/f: Empty file

$ clamscan -r --exclude-dir=^/e/ d
d/e/g: Empty file
d/f: Empty file

$ mkdir d/e2/e5 d/3e3/4e4
$ touch d/{e,e2,3e3}/{e5,4e4}f

$ clamscan -r --exclude-dir=^/e/ d
d/3e3/4e4f: Empty file
d/3e3/e5f: Empty file
d/3e3/f: Empty file
d/e/4e4f: Empty file
d/e/e5f: Empty file
d/e/f: Empty file
d/e/g: Empty file
d/e2/4e4f: Empty file
d/e2/e5f: Empty file
d/e2/f: Empty file
d/f: Empty file

$ clamscan -r --exclude-dir=d/e d
d/3e3/4e4f: Empty file
d/3e3/e5f: Empty file
d/3e3/f: Empty file
d/e: Excluded
d/e2: Excluded
d/f: Empty file

$ clamscan -r --exclude=d/e d
d/3e3/4e4f: Empty file
d/3e3/e5f: Empty file
d/3e3/f: Empty file
d/e/4e4f: Excluded
d/e/e5f: Excluded
d/e/f: Excluded
d/e/g: Excluded
d/e2/4e4f: Excluded
d/e2/e5f: Excluded
d/e2/f: Excluded
d/f: Empty file

$ clamscan -r --exclude=/tmp/d/e d
d/3e3/4e4f: Empty file
d/3e3/e5f: Empty file
d/3e3/f: Empty file
d/e/4e4f: Empty file
d/e/e5f: Empty file
d/e/f: Empty file
d/e/g: Empty file
d/e2/4e4f: Empty file
d/e2/e5f: Empty file
d/e2/f: Empty file
d/f: Empty file

$ clamscan -r --exclude=/tmp/d/e /tmp/d
/tmp/d/3e3/4e4f: Empty file
/tmp/d/3e3/e5f: Empty file
/tmp/d/3e3/f: Empty file
/tmp/d/e/4e4f: Excluded
/tmp/d/e/e5f: Excluded
/tmp/d/e/f: Excluded
/tmp/d/e/g: Excluded
/tmp/d/e2/4e4f: Excluded
/tmp/d/e2/e5f: Excluded
/tmp/d/e2/f: Excluded
/tmp/d/f: Empty file

$ clamscan -r --exclude=^/tmp/d/e /tmp/d
/tmp/d/3e3/4e4f: Empty file
/tmp/d/3e3/e5f: Empty file
/tmp/d/3e3/f: Empty file
/tmp/d/e/4e4f: Excluded
/tmp/d/e/e5f: Excluded
/tmp/d/e/f: Excluded
/tmp/d/e/g: Excluded
/tmp/d/e2/4e4f: Excluded
/tmp/d/e2/e5f: Excluded
/tmp/d/e2/f: Excluded
/tmp/d/f: Empty file

$ clamscan -r --exclude-dir=/.e d
d/3e3: Excluded
d/e/4e4f: Empty file
...

$ clamscan -r --exclude-dir=/*e d
d/3e3: Excluded
d/e: Excluded
d/e2: Excluded
d/f: Empty file

$ clamscan -r --exclude-dir=/3* d
d/3e3: Excluded
d/e: Excluded
d/e2: Excluded
d/f: Empty file

$ mkdir d/p\ a
$ touch d/p\ a/file
$ clamscan -r --exclude-dir=\\s+ d
d/3e3/4e4f: Empty file
d/3e3/e5f: Empty file
d/3e3/f: Empty file
d/e/4e4f: Empty file
d/e/e5f: Empty file
d/e/f: Empty file
d/e/g: Empty file
d/e2/4e4f: Empty file
d/e2/e5f: Empty file
d/e2/f: Empty file
d/p a/file: Empty file
d/f: Empty file

$ clamscan -r --exclude-dir='\s+' d
d/3e3/4e4f: Empty file
d/3e3/e5f: Empty file
d/3e3/f: Empty file
d/e/4e4f: Empty file
d/e/e5f: Empty file
d/e/f: Empty file
d/e/g: Empty file
d/e2/4e4f: Empty file
d/e2/e5f: Empty file
d/e2/f: Empty file
d/p a/file: Empty file
d/f: Empty file

“Scripting” is not sloppy “programming”

Every time I need to mention some automation development work while talking to a client, I find myself overly watchful not to say the words “scripting” or “script” excessively. That’s how much of “sloppiness” is felt to be connotative to those terms in the public opinion.

Ergo, you can see how glad I was to find in the Chapter 7 of the “UNIX and Linux System Administration Handbook” this understanding that

There’s no real distinction between “scripting” and “programming.” Language developers sometimes take offense when their babies are lumped into the “scripting” category, not just because the label suggests a certain lack of completeness, but also because some scripting languages of the past have earned reputations for poor design.

Of course, the importance of scripting cannot be overestimated as it is the main mediator in performing administrative changes reliably, in a consistent and repeatable manner, which are so important in the system administrator’s job as in no other “soft” profession.

Speaking of the subtle differences between “scripting” and “programming”, one could point out the issue of computing system efficiency. Indeed, “scripted” procedures are often several times “slower” than “programmed” counterparts. Administrative work, however, is more about the effectiveness of the administrator, which is the efficiency of the binding administrator+computing_system. As the book authors put it,

Optimization can have an amazingly low return on investment, even for scripts that run regularly out of cron.

Which is more significant out of the two Linus Torvalds’ biggest invetions: Linux kernel of Git? – another great question from the book. On one hand, Linux (albeit not the kernel alone) in this very 5th edition of the book has ousted all the other Unix systems of other editions, including commercial ones – so, perhaps it is quite important. On the other hand,

Mistakes are a fact of life.

Shortest monitoring script on earth

Right after I got my quite elaborate Nagios setup running in my local network, I thought to myself “What if my network loses connectivity to the world and my GSM-SMS gateway fails? How am I going to be notified of such a grand disaster?”

Well, as usual, a shell one-liner comes to rescue.

Multiple –exclude options to tar

The tar’s “–exclude=PATTERN” option has always had me perplexed. As this is mostly intended for automation scenarios, I have never paid much attention to this option resorting to its more “pro-batch” variant “–exclude-from FILE”.

But today, as I went about making some backups I thought it’s enough, I need to learn how to use this option effectively. Having realized that the documentation on the matter is quite lacking, I decided to make a simple experiment.

Here are the commands I typed (in a bash terminal) to learn more about the “–exclude=PATTERN” option:
Read More