Skip to content

123Unix!com

Easy Automation is here

  • About
  • Ask a Question
  • Automation Services
  • About
  • Ask a Question
  • Automation Services

Yearly Archives: 2020

  • Home
  • 2020

clamscan exclude & include REGEX format

clamscan documentation is silent about which format the application expects for the –exclude=REGEX, –exclude-dir=REGEX, –include=REGEX, –include-dir=REGEX command line options.

Here are the REGEX rules found experimentally:

  1. . is any symbol
  2. * means zero or more occurrences of the preceding character
  3. + means 1 or more occurrences of the preceding character
  4. ^ ancors to the start of the filename or directory name with full path, either absolute or relative to CWD
  5. / is treated literally as a path subdirectories separator
  6. leading / ancors to the beginning of the filename or directory name
  7. trailing / ancors to the end of the directory name
  8. \s \w \S and other escapes seem to have the backslash ignored
  9. buy this point I got bored 🙂

What follows is the listing of the experiments shell session.

$ pwd
/tmp
$ mkdir d
$ mkdir d/e
$ touch d/f d/e/g
$ clamscan d
d/f: Empty file

$ clamscan --exclude-dir=f d
d/f: Empty file

$ clamscan -r --exclude-dir=f d
d/e/g: Empty file
d/f: Empty file

$ clamscan -r --exclude-dir=e d
d/e: Excluded
d/f: Empty file

$ clamscan -r --exclude-dir=/e d
d/e: Excluded
d/f: Empty file

$ mkdir d/e2 d/3e3 
$ clamscan -r --exclude-dir=/e d
d/e: Excluded
d/e2: Excluded
d/f: Empty file

$ clamscan -r --exclude-dir=/e/ d
d/e: Excluded
d/f: Empty file

$ clamscan -r --exclude-dir=^/e/ d
d/e/g: Empty file
d/f: Empty file

$ mkdir d/e2/e5 d/3e3/4e4
$ touch d/{e,e2,3e3}/{e5,4e4}f

$ clamscan -r --exclude-dir=^/e/ d
d/3e3/4e4f: Empty file
d/3e3/e5f: Empty file
d/3e3/f: Empty file
d/e/4e4f: Empty file
d/e/e5f: Empty file
d/e/f: Empty file
d/e/g: Empty file
d/e2/4e4f: Empty file
d/e2/e5f: Empty file
d/e2/f: Empty file
d/f: Empty file

$ clamscan -r --exclude-dir=d/e d
d/3e3/4e4f: Empty file
d/3e3/e5f: Empty file
d/3e3/f: Empty file
d/e: Excluded
d/e2: Excluded
d/f: Empty file

$ clamscan -r --exclude=d/e d
d/3e3/4e4f: Empty file
d/3e3/e5f: Empty file
d/3e3/f: Empty file
d/e/4e4f: Excluded
d/e/e5f: Excluded
d/e/f: Excluded
d/e/g: Excluded
d/e2/4e4f: Excluded
d/e2/e5f: Excluded
d/e2/f: Excluded
d/f: Empty file

$ clamscan -r --exclude=/tmp/d/e d
d/3e3/4e4f: Empty file
d/3e3/e5f: Empty file
d/3e3/f: Empty file
d/e/4e4f: Empty file
d/e/e5f: Empty file
d/e/f: Empty file
d/e/g: Empty file
d/e2/4e4f: Empty file
d/e2/e5f: Empty file
d/e2/f: Empty file
d/f: Empty file

$ clamscan -r --exclude=/tmp/d/e /tmp/d
/tmp/d/3e3/4e4f: Empty file
/tmp/d/3e3/e5f: Empty file
/tmp/d/3e3/f: Empty file
/tmp/d/e/4e4f: Excluded
/tmp/d/e/e5f: Excluded
/tmp/d/e/f: Excluded
/tmp/d/e/g: Excluded
/tmp/d/e2/4e4f: Excluded
/tmp/d/e2/e5f: Excluded
/tmp/d/e2/f: Excluded
/tmp/d/f: Empty file

$ clamscan -r --exclude=^/tmp/d/e /tmp/d
/tmp/d/3e3/4e4f: Empty file
/tmp/d/3e3/e5f: Empty file
/tmp/d/3e3/f: Empty file
/tmp/d/e/4e4f: Excluded
/tmp/d/e/e5f: Excluded
/tmp/d/e/f: Excluded
/tmp/d/e/g: Excluded
/tmp/d/e2/4e4f: Excluded
/tmp/d/e2/e5f: Excluded
/tmp/d/e2/f: Excluded
/tmp/d/f: Empty file

$ clamscan -r --exclude-dir=/.e d
d/3e3: Excluded
d/e/4e4f: Empty file
...

$ clamscan -r --exclude-dir=/*e d
d/3e3: Excluded
d/e: Excluded
d/e2: Excluded
d/f: Empty file

$ clamscan -r --exclude-dir=/3* d
d/3e3: Excluded
d/e: Excluded
d/e2: Excluded
d/f: Empty file

$ mkdir d/p\ a
$ touch d/p\ a/file
$ clamscan -r --exclude-dir=\\s+ d
d/3e3/4e4f: Empty file
d/3e3/e5f: Empty file
d/3e3/f: Empty file
d/e/4e4f: Empty file
d/e/e5f: Empty file
d/e/f: Empty file
d/e/g: Empty file
d/e2/4e4f: Empty file
d/e2/e5f: Empty file
d/e2/f: Empty file
d/p a/file: Empty file
d/f: Empty file

$ clamscan -r --exclude-dir='\s+' d
d/3e3/4e4f: Empty file
d/3e3/e5f: Empty file
d/3e3/f: Empty file
d/e/4e4f: Empty file
d/e/e5f: Empty file
d/e/f: Empty file
d/e/g: Empty file
d/e2/4e4f: Empty file
d/e2/e5f: Empty file
d/e2/f: Empty file
d/p a/file: Empty file
d/f: Empty file

  • 20 Mar, 2020
  • (0) Comments
  • By Alex
  • /var/log, How-to, Tools

Developer of Operations

When talking about Configuration Management it is important to acknowledge that writing Configuration Management code, or implementing “IaC” is virtually the same kind of work the software developers are performing. You use a professional code editor, abide to a strict set of syntax rules, track changes in Git or another VCS.

As an admin going the Configuration Management way, you effectively become a Developer of Operations for your infrastructure.

To emphasize the significance of the shift in work nature, the authors of the “UNIX and Linux System Administration Handbook” say that approaching CM activities in a different manner, without coordination and software developer attitude will surely result in “a muddle of conflicting or parallel code” and ultimately “the abyss”.

Other notable quotes from the chapter are these:

  • changes should be structured, automated, and applied consistently among machines.
  • People sometimes conflate DevOps and configuration management
  • snowflake model of system administration – when no two systems are ever alike.
  • security implications of this approach, however: the client controls the facts that it reports, so make sure that a compromised client can’t exploit the configuration management system to gain additional privileges.
  • Nothing that runs out of cron, for example, can depend on the presence of an administrator to enter passwords. Working around that constraint inevitably ends up lowering security to the level of the root account.
  • all major CM systems use similar conceptual models,
    • they describe these models with different lexicons.
    • Unfortunately, the terminology used by a particular CM system often has more to do with conforming to a marketing theme than with maximizing clarity.
  • “environments,” both inside and outside the configuration management context.
    • This seems to be the single term on which all configuration management systems agree.
    • They’re an additional axis of variation that can affect multiple aspects of the configuration.
  • A number of projects focus on specific subdomains of configuration management,
    • notably new-system provisioning (e.g., Cobbler) and
    • software deployment (e.g., Fabric and Capistrano)

  • 12 Feb, 2020
  • (0) Comments
  • By Alex
  • Training

Servers are cattle, not pets

This is the essence of the Chapter 9 “Cloud Computing” of the “UNIX and Linux System Administration Handbook.”

Indeed, you need to treat cloud “servers” as disposable, otherwise it is going to cost you more than your existing in-house server fleet – just like with real cattle it would be more expensive to keep even a small herd at home compared to a single, albeit rare-breed super dog.

On other notes:

  • when moving into cloud, system administrator’s role shrinks considerably:
    • – administrators’ duties are shifted from in-house’ to in-cloud’s admins
      • 1) from local servers to IaaS: half (5/10) of the layers to administer are dropped
      • 2) from IaaS to PaaS: another 2/5 is dropped
      • 3) in SaaS virtually no work is left for the in-company system administrator
    • automation
      • – AWS CloudFormation (native, error prone)
      • – Troposphere Python library – eases CF
      • – Terraform

    And a few quotes from the book:

    – servers should be treated as cattle, not as pets

    – A system is said to be cloud native if it is reliable even in the face of unanticipated events.

    – hybrid cloud
    — operating two distinct cloud presences in tandem increases complexity more than proportionally.

    – AWS offers on-site visits from the AWS Snowmobile, a 45-foot long shipping container towed by a semi truck than can transfer 100 PiB from your data center to the cloud.

    – access controls should conform to the principle of least privilege

    – Serverless functions hold great promise for the industry.
    — AWS introduced Lambda, their cloud function service, at a conference in 2014.
    — Google followed shortly with a Cloud Functions service.
    — Several cloud function implementations exist for projects like OpenStack, Mesos, and Kubernetes.

    – Costs
    — new cloud customers are often surprised when costs climb quickly.
    — third party cloud use analyze services to optimize overprovisioning:
    — Cloudability
    — CloudHealth

  • 31 Jan, 2020
  • (0) Comments
  • By Alex
  • Training