--------------------------------------------------------------
The exiscan-acl patch for exim4 - Documentation
--------------------------------------------------------------
(c) Tom Kistner <tom@duncanthrax.net> 2003-????
License: GPL

The exiscan-acl patch adds  content scanning to the  exim4 ACL
system. It supports the following scanning facilities:

 - MIME unpacking, sanity checking, file extension blocking
 - Antivirus using 3rd party scanners
 - Antispam using SpamAssassin
 - Regular expression match against headers and body
 
These facilities are hooked into exim by adding new conditions
to exim's ACL system. These conditions are designed to be used
in the acl_smtp_data ACL. It is run when the sending host  has
completed the DATA phase and is waiting for our final response
to his end-of-data marker. This allows  us to  reject messages
containing unwanted content at that stage.

The exiscan-acl patch also defines several expansion variables
that can be used to customise the error responses sent to  the
remote server.

The   default   exim   configure   file   contains   commented
configuration examples for all facilites.


0. Overall concept
--------------------------------------------------------------

The   exiscan-acl   patch   adds   the   following  conditions
(facilities), which  can ONLY  be used  in the  ACL after DATA
(acl_smtp_data):

- demime (MIME unpacking and file extension checks)
- regex (match regular expressions against message headers
  and body)
- malware (attach 3rd party virus/malware scanner)
- spam (attach SpamAssassin)

Each of these facilities has its own chapter further below  in
this document. There is also a commented  sample configuration
in the "configure" file of the exim distribution.

The exiscan  facility also  adds the  'fakereject' ACL control
statement that can be used  to reject and accept a  message at
the same time. It is described in an extra chapter below.

All  facilites work  on a  MBOX copy  of the  message that  is
temporarily spooled up in a file called:

  <spool_directory>/scan/<message_id>/<message_id>.eml

The .eml extension is a  friendly hint to virus scanners  that
they can expect an  MBOX-like structure inside that  file. The
file is only spooled up once, when the first exiscan  facility
condition is  called. Subsequent  calls to  exiscan conditions
will just open  the file again.  The directory is  recursively
removed when the acl_smtp_data has finished running. When  the
"demime" condition  has been  used, this  directory will  also
contain files produced by the MIME decoder.


1. The "demime" facility
   MIME unpacking, sanity checking and file extension blocking
--------------------------------------------------------------

The demime facility unpacks MIME containers in the message. It
detects  errors  in  MIME   containers  and  can  match   file
extensions found  in the  message against  a list.  Using this
facility will produce additional  files in the temporary  scan
directory that contain the unpacked MIME parts of the message.
If you  do antivirus  scanning, it  is recommened  to use  the
"demime" condition before the antivirus ("malware") condition.

The condition name of this facility is "demime". On the  right
hand  side,  you  can  pass  a  colon-separated  list  of file
extensions that it  should match against.  If one of  the file
extensions  is  found,  the  condition  will  return  "OK" (or
"true"), otherwise it will return FAIL (or "false"). If  there
was any TEMPORARY error while demimeing (mostly "disk  full"),
the  condition  will return  DEFER,  and the  message  will be
temporarily rejected.

The right-hand side gets "expanded" before being treated as  a
list, so  you can  have conditions  and lookups  there. If  it
expands  to  an  empty  string,  "false",  or  zero  ("0"), no
demimeing is done and the conditions returns FALSE.

A short example:

/* ------------
deny message = Found blacklisted file attachment
     demime = vbs:com:bat:pif:prf:lnk
--------------- */

When the condition is run, it sets up the following  expansion
variables:

 $demime_errorlevel   When  an error  was detected  in a  MIME
                      container, this   variable contains  the
                      "severity"  of the error,  as an integer
                      number.  The   higher  the   value,  the
                      more  severe    the  error.    If   this
                      variable is unset or zero, no error  has
                      occured.

 $demime_reason       When $demime_errorlevel is greater  than
                      zero,  this  variable  contains  a human
                      -readable  text  string  describing  the
                      MIME error that occured.
                          
 $found_extension     When  the  "demime"  condition   returns
                      "true", this variable contains the  file
                      extension it has found.
                          
Both $demime_errorlevel  and $demime_reason  are set  with the
first call of the "demime"  condition, and are not changed  on
subsequent calls.

If do not  want to check  for any file  extensions, but rather
use  the  demime  facility  for  unpacking  or  error checking
purposes, just pass "*" as the right-hand side value.

Here is a more elaborate example on how to use this facility:

/* -----------------
# Reject messages with serious MIME container errors
deny  message = Found MIME error ($demime_reason).
      demime = *
      condition = ${if >{$demime_errorlevel}{2}{1}{0}}

# Reject known virus spreading file extensions.
# Accepting these is pretty much braindead.
deny  message = contains $found_extension file (blacklisted).
      demime = com:vbs:bat:pif:scr

# Freeze .exe and .doc files. Postmaster can
# examine them and eventually thaw them up.
deny  log_message = Another $found_extension file.
      demime = exe:doc
      control = freeze
--------------------- */



2. The "spam" facility
   Antispam measures with SpamAssassin
--------------------------------------------------------------

The "spam" facility calls SpamAssassin's "spamd" daemon to get
a spam-score  and a  report for  the message.  You must  first
install     SpamAssassin.     You     can     get     it    at
http://www.spamassassin.org, or,  if you  have a  working Perl
installation, you can use CPAN by calling

perl -MCPAN -e 'install Mail::SpamAssassin'

SpamAssassin has it's own  set of configuration files.  Please
review its  documentation to  see how  you can  tweak it.  The
default installation should work nicely, however.

After having installed and configured SpamAssassin, start  the
"spamd" daemon. By default, it  listens on 127.0.0.1, TCP port
783. If you use another host  or port for spamd, you must  set
the   spamd_address  option   in  Section   1  of   the  exim
configuration as follows (example):

spamd_address = 127.0.0.1 783

If you use the above mentioned default, you do NOT need to set
this option.

To use the  antispam facility, put  the "spam" condition  in a
DATA ACL block. Here is a very simple example:

/* ---------------
deny message = This message was classified as SPAM
        spam = joe
---------------- */

On the right-hand side of the spam condition, you can put  the
username that SpamAssassin should scan for. That allows you to
use per-domain or  per-user antispam profiles.  The right-hand
side is expanded before being used, so you can put lookups  or
conditions there. When the right-hand side evaluates to "0" or
"false", no scanning will be done and the condition will  fail
immediately.

If you do not want to  scan for a particular user, but  rather
use the SpamAssassin system-wide default profile, you can scan
for an unknown user, or simply use "nobody".

The  "spam"  condition  will  return  true  if  the  threshold
specified in the user's SpamAssassin profile has been  matched
or exceeded. If  you want to  use the spam  condition for it's
side effects (see the variables below), you can make it always
return "true" by appending ":true" to the username. 

When the condition is run, it sets up the following  expansion
variables:

  $spam_score       The spam score of the message, for example
                    "3.4"  or  "30.5".  This  is  useful   for
                    inclusion in log or reject messages.
                  
  $spam_score_int   The spam score of the message,  multiplied
                    by ten, as  an integer value.  For example
                    "34" or "305". This is useful for  numeric
                    comparisons  in  conditions.  See  further
                    below for a more complicated example. This
                    variable is special,  since it is  written
                    to  the  spool  file, so  it  can  be used
                    during the  whole life  of the  message on
                    your exim system, even in routers
                    or transports.
                    
  $spam_bar         A string consisting of a number of '+'  or
                    '-'    characters,    representing     the
                    spam_score value.  A spam  score of  "4.4"
                    would have a  spam_bar of '++++'.  This is
                    useful for  inclusion in  warning headers,
                    since MUAs can match on such strings.
  
  $spam_report      A  multiline  text  table,  containing the
                    full SpamAssassin report for the  message.
                    Useful for inclusion in headers or  reject
                    messages.
            
The spam condition  caches its results.  If you call  it again
with the same  user name, it  will not really  scan again, but
rather return the same values as before.
                    
Finally, here is  a commented example  on how to  use the spam
condition:

/* ----------------
# put headers in all messages (no matter if spam or not)
warn  message = X-Spam-Score: $spam_score ($spam_bar)
      spam = nobody:true
warn  message = X-Spam-Report: $spam_report
      spam = nobody:true
      
# add second subject line with *SPAM* marker when message
# is over threshold
warn  message = Subject: *SPAM* $h_Subject
      spam = nobody

# reject spam at high scores (> 12)
deny   message = This message scored $spam_score spam points.
       spam = nobody:true
       condition = ${if >{$spam_score_int}{120}{1}{0}}
----------------- */



3. The "regex" facility
   Match headers and body lines of the message against regular
   expressions
--------------------------------------------------------------

The "regex" condition takes one or more regular expressions as
arguments and matches them  against the full message,  that is
all headers and the  complete body. It is  particularly useful
to  filter trash  that cannot  be recognized  by the  spam  or
malware conditions. With large messages, this condition can be
fairly CPU-intensive.

The regular expressions are  matched linewise, with a  maximum
line length of 32k characters.

The regular expressions are passed as a colon-separated  list.
To include  a literal  colon, you  must double  it. Since  the
whole right-hand  side string  is expanded  before being used,
you must also escape dollar ($) signs with backslashes.

Here is a simple example:

/* ----------------------
deny message = contains blacklisted regex ($regex_match_string)
     regex = [Mm]ortgage : URGENT BUSINESS PROPOSAL
----------------------- */

The condition returns true  if one of the  regular expressions
has matched  a line  of the  message. The  $regex_match_string
variable  is then  set up  and contains  the matching  regular
expression.



4. The "malware" facility
   Scan messages for viruses using an external virus scanner
--------------------------------------------------------------

This facility lets you connect virus scanner software to exim.
It supports a "generic"  interface to scanners called  via the
shell,  and  specialized interfaces  for  "daemon" type  virus
scanners, who are resident in memory and thus are much faster.

To use this facility, you MUST set the "av_scanner" option  in
section 1 of  the exim config  file. It specifies  the scanner
type to use, and any  additional options it needs to  run. The
basic syntax is as follows:

  av_scanner = <scanner-type>:<option1>:<option2>:[...]
  
The following scanner-types are supported in this release:

  sophie      Sophie  is a  daemon that  uses Sophos'  libsavi
              library to scan for viruses. You can get  Sophie
              at http://www.vanja.com/tools/sophie/. The  only
              option for this scanner type is the path to  the
              UNIX  socket   that  Sophie   uses  for   client
              communication.    The     default    path     is
              /var/run/sophie, so if  you are using  this, you
              can omit the option. Example:
              
              av_scanner = sophie:/tmp/sophie


  kavdaemon   Kapersky's kavdaemon  is a  daemon-type scanner.
              You    can    get    a    trial    version    at
              http://www.kapersky.com. This scanner type takes
              one option,  which is  the path  to the daemon's
              UNIX socket.  The default  is "/var/run/AvpCtl".
              Example:
              
              av_scanner = kavdaemon:/opt/AVP/AvpCtl
              
              
  clamd       Another daemon type scanner, this one is GPL and
              free. Get  it at  http://clamav.elektrapro.com/.
              Clamd does not  seem to unpack  MIME containers,
              so it is recommended to use the demime  facility
              with it.  It takes  one option:  either the path
              and  name   of  a   UNIX  socket   file,  or   a
              hostname/port  pair,  separated  by  space.   If
              unset, the default is "/tmp/clamd". Example:
              
              av_scanner = clamd:192.168.2.100 1234
              or
              av_scanner = clamd:/opt/clamd/socket
              
              
  drweb       This one is for the DrWeb (http://www.sald.com/)
              daemon.  It takes  one argument,  either a  full
              path to a UNIX socket, or an IP address and port
              separated  by  whitespace.   If  you  omit   the
              argument, the default
              
              /usr/local/drweb/run/drwebd.sock
              
              is used. Example:
              
              av_scanner = drweb:192.168.2.20 31337
              or
              av_scanner = drweb:/var/run/drwebd.sock
              
              Thanks  to  Alex  Miller  <asm@abbyy.com.ua> for
              contributing the code for this scanner.              
              
              
  mksd        Yet another daemon type scanner, aimed mainly at
              Polish users, though some parts of documentation
              are now avaliable in English.  You can get it at
              http://linux.mks.com.pl/.  The only  option  for
              this  scanner  type  is the  maximum  number  of
              processes   used   simultaneously  to  scan  the
              attachments, provided  that the  demime facility
              is  employed  and also  mksd has been  run  with
              at least  the same  number of  child  processes.
              You can  safely  omit this  option,  the default
              value is 1. Example:
              
              av_scanner = mksd:2


  cmdline     This is the keyword for the generic command line
              scanner  interface.  It can  be  used to  attach
              virus scanners  that are  invoked on  the shell.
              This scanner type takes 3 mantadory options:
              
              - full path and name of the scanner binary, with
                all  command  line options  and  a placeholder
                (%s) for the directory to scan.
                
              - A  regular  expression  to  match  against the
                STDOUT and STDERR output of the virus scanner.
                If the expression matches, a virus was  found.
                You  must  make  absolutely  sure  that   this
                expression only matches on "virus found". This
                is called the "trigger" expression.
                
              - Another regular expression, containing exactly
                ONE pair of braces,  to match the name  of the
                virus found  in the  scanners output.  This is
                called the "name" expression.
                
              Example:
              
              Sophos  Sweep reports  a virus  on a  line  like
              this:

              Virus 'W32/Magistr-B' found in file ./those.bat

              For the  "trigger" expression,  we just  use the
              "found" word. For the "name" expression, we want
              to get the W32/Magistr-B string, so we can match
              for  the single  quotes left  and right  of it,
              resulting in the regex '(.*)' (WITH the quotes!)

              Altogether,   this   makes   the   configuration
              setting:
              
              av_scanner = cmdline:\
              /path/to/sweep -all -rec -archive %s:\
              found:'(.+)'
              

When av_scanner  is correcly  set, you  can use  the "malware"
condition in the  DATA ACL. The  condition takes a  right-hand
argument that is expanded before use. It can then be one of

  - "true", "*", or "1", in which case the message is  scanned
    for viruses.  The condition  will succeed  if a  virus was
    found, or fail otherwise. This is the recommended usage.
    
  - "false" or "0", in which case no scanning is done and  the
    condition will fail immediately.
    
  - a regular expression, in which case the message is scanned
    for viruses. The condition  will succeed if a  virus found
    found and  its name  matches the  regular expression. This
    allows you  to take  special actions  on certain  types of
    viruses.
    
When a  virus was  found, the  condition sets  up an expansion
variable called  $malware_name that  contains the  name of the
virus found. You  should use it  in a "message"  modifier that
contains the error returned to the sender.

The malware condition caches its  results, so when you use  it
multiple times,  the actual  scanning process  is only carried
out once.

If your virus scanner  cannot unpack MIME and  TNEF containers
itself,  you  should use  the  demime condition  prior  to the
malware condition.

Here is a simple example:

/* ----------------------
deny message = This message contains malware ($malware_name)
     demime = *
     malware = *
---------------------- */
     


5. The "fakereject" control statement
   Reject a message while really accepting it.
--------------------------------------------------------------

When you put "control =  fakereject" in an ACL statement,  the
following  will  happen:  If  exim  would  have  accepted  the
message, it will tell the remote host that it did not, with  a
message of:

550-FAKE_REJECT id=xxxxxx-xxxxxx-xx
550-Your message has been rejected but is being kept for evaluation.
550 If it was a legit message, it may still be delivered to the target recipient(s).

But exim will go on to treat the message as if it had accepted
it. This should be used with extreme caution, please look into
the examples document for possible usage.



--------------------------------------------------------------
End of file
--------------------------------------------------------------
