dccm(8)

NAME
dccm - Distributed Checksum Clearinghouse Milter Interface

SYNOPSIS
dccm [-VdbxANQ] [-G on | off | noIP | IPmask/xx] [-h homedir] [-I user]
[-p protocol:filename | protocol:port@host] [-m map]
[-w whiteclnt] [-U userdirs] [-a IGNORE | REJECT | DISCARD]
[ -t type, [log-thold,] rej-thold] [ -g [not-] type] [-S header]
[-l logdir] [-R rundir] [-r rejection-msg] [-j maxjobs] [-B dnsbl-option] [-L ltype,facility.level]

DESCRIPTION
dccm is a daemon built with the sendmail milter interface intended to connect sendmail(8) to DCC servers. When built with the milter filter machinery and configured to talk to dccm in the sendmail.cf file, sendmail passes all email to dccm which in turn reports related checksums to the nearest DCC server. dccm then adds an X-DCC SMTP header line to the message. Sendmail is told to reject the message if it is unsolicited bulk mail.

Dccm sends reports of checksums related to mail received by DCC clients and queries about the total number of reports of particular checksums. A DCC server receives no mail, address, headers, or other information, but only cryptographically secure checksums of such information. A DCC server cannot determine the text or other information that corresponds to the checksums it receives. Its only acts as a clearinghouse of counts for checksums computed by clients. For complete privacy as far as the DCC is concerned, the checksums of purely internal mail or other mail that is known to not be unsolicited bulk can be listed in a whitelist to not be reported to the DCC server.

Since the checksums of messages that are whitelisted locally by the -w whiteclnt file are not reported to the DCC server, dccm knows nothing about the total recipient counts for their checksums and so cannot add X-DCC header lines to such messages. Sendmail does not tell dccm about messages that are not received by sendmail via SMTP, including messages submitted locally and received via UUCP, and so they also do not receive X-DCC header lines.

Enable the daemon and put its parameters in the dcc_/usr/local/dcc/conf file and start the daemon with the /usr/local/dcc/libexec/start-dccm or var/dcc/libexec/rcDCC scripts.

The list of servers that dccm contacts is in the memory mapped file map shared by local DCC clients. The file is maintained with cdcc(8).

  OPTIONS
The following options are available:

-V  displays the version of dccm.

-d  enables debugging output from the DCC client software. Additional -d options increase the number of messages. A single -d logs aborted SMTP transactions including those from some "dictionary attacks."

-b  causes the daemon to not detach itself from the controlling tty and put itself into the background.

-x  causes the daemon to try "extra hard" to contact a DCC server. Since it is usually more important to deliver mail than to report its checksums, dccm normally does not delay too long while trying to contact a DCC server. It will not try again for several seconds after a failure. With -x, it will always try to contact the DCC server and it will tell the MTA to answer the DATA command with a 4yz temporary failure.

-A  adds to existing X-DCC headers in the message instead of replacing existing headers of the brand of the current server.

-N  neither adds, deletes, nor replaces existing X-DCC headers in the message. Each message is logged, rejected, and otherwise handled the same.

-Q  only queries the DCC server about the checksums of messages instead of reporting and querying. This is useful when dccm is used to filter mail that has already been reported to a DCC server by another DCC client. No single mail message should be reported to a DCC server more than once per recipient, because each report will increase the apparent "bulkness" of the message.

    It is better to use MXDCC lines in the global whiteclnt file for your MX mail servers that use DCC than -Q.

-G on | off | noIP | IPmask/xx
    controls greylisting. At least one working greylist server must be listed in the map file in the DCC home directory. If more than one is named, they must "flood" or change checksums and they must use the same -G parameters. See dccd(8). Usually all dccm or dccifd DCC client processes use the same -G parameters.

    IPmask/xx and noIP remove part or all of the IP address from the greylist triple.

-h homedir
    overrides the default DCC home directory, /usr/local/dcc.

-I user
    specifies the UID and GID of the process.

-p protocol:filename | protocol:port@host
    specifies the protocol and address by which sendmail will contact dccm. The default is a UNIX domain socket in the "run" directory, /var/run/dcc/dccm. (See also -R) This protocol and address must match the value in sendmail.cf. This mechanism can be used to connect dccm on one computer to sendmail on another computer when a port and host name or IP address are used.

-m map
    specifies a name or path of the memory mapped parameter file instead of the default map file in the DCC home directory. It should be created with the cdcc(8) command.

-w whiteclnt
    specifies an optional file containing filtering parameters as well as SMTP client IP addresses, SMTP envelope values, and header values of mail that is spam or is not spam and does not need a X-DCC header, and whose checksums should not be reported to the DCC server.

    If the pathname whiteclnt is not absolute, it is relative to the DCC home directory.

    The format of the dccm whiteclnt file is the same as the whitelist files used by dbclean(8) and the whiteclnt file used by dccproc(8). See dcc(8) for a description of DCC white and blacklists. Because the contents of the whiteclnt file are used frequently, a companion file is automatically created and maintained. It has the same pathname but with an added suffix of .dccw and contains a memory mapped hash table of the main file.

    A whitelist entry ("OK") or two or more semi-whitelistings ("OK2") for one of the message's checksums prevents all of the message's checksums from being reported to the DCC server and the addition of a X-DCC header line by dccm A whitelist entry for a checksum also prevents rejecting or discarding the message based on DCC recipient counts as specified by -a and -t. Otherwise, one or more checksums with blacklisting entries ("MANY") cause all of the message's checksums to be reported to the server with an addressee count of "MANY".

    If the message has a single recipient, an env_To whiteclnt entry of "OK" for the checksum of its recipient address acts like any other whiteclnt entry of "OK." When the SMTP message has more than one recipient, the effects can be complicated. When a message has several recipients with some but not all listed in the whiteclnt file, dccm tries comply with the wishes of the users who want filtering as well as those who don't by silently not delivering the message to those who want filtering (i.e. are not whitelisted) and delivering the message to don't want filtering.

-U userdirs
    enables per-user whiteclnt files and log directories. Each target of a message can have a directory of log files named usedirs/${dcc_userdir}/log where ${dcc_userdir} is the sendmail.cf macro described below. If ${dcc_userdir} is not set, userdirs/${rcpt_mailer}/${rcpt_addr}/log is used. The most likely value of mailer is local. Appropriate values for both ${rcpt_mailer} and ${rcpt_addr} can be seen by examining env_To lines in -l logdir files. If it is not absolute, userdirs is relative to the DCC home directory. The directory containing the log files must be named log and it must be writable by the dccm process. Each log directory must exist or logging for the corresponding is silently disabled. The files created in the log directory are owned by the UID of the dccm process, but they have group and other read and write permissions copied from the corresponding log directory. To ensure the privacy of mail, it may be good to make the directories readable only by owner and group, and to use a cron script that changes the owner of each file to match the grandparent addr directory.

    There can also be a per -user whitelist file named userdirs/${dcc_userdir}/whiteclnt or if ${dcc_userdir} is not set, userdirs/${rcpt_mailer}/${rcpt_addr} per-user whitelist files. Any checksum that is not white- or blacklisted by an individual addressee's whiteclnt file is checked in the main -w -whiteclnt file. A missing per-addressee whiteclnt file is the same as an empty file. Relative paths for files included in per-addressee files are resolved in the DCC home directory. The whiteclnt files and the addr directories containing them must be writable by the dccm process.

    Option lines in per-user whiteclnt files can be used to modify many aspects of dccm filtering, as described in the main dcc man page. For example, an option dcc-off line turns off DCC filtering for individual mailboxes.

-a IGNORE | REJECT | DISCARD
    specifies the action taken when DCC server counts or -t thresholds say that a message is unsolicited and bulk. IGNORE causes the message to be unaffected except for adding the X-DCC header line to the message. This turns off DCC filtering.

    Spam can also be REJECTed or accepted and silently DISCARDed without being delivered to local mailboxes. The default is REJECT.

    Mail forwarded via IP addresses marked MX or MXDCC in the main whiteclnt file is treated as if -a DISCARD were specified. This prevents "bouncing" spam.

    Determinations that mail is or is not spam from sendmail via ${dcc_isspam} or ${dcc_notspam} macros override -a. The effects of the -w whiteclnt are not affected by -a.

-t  type, [log-thold,] rej-thold sets logging and "spam" thresholds for checksum type. The checksum types are IP, env_From, From, Message-ID, substitute, Received, Body, Fuz1, Fuz2, rep-total, and rep. The first six, IP through substitute, have no effect except when a local DCC server configured with -K is used. The substitute thresholds apply to the first substitute heading encountered in the mail message. The string ALL sets thresholds for all types, but is unlikely to be useful except for setting logging thresholds. The string CMN specifies the commonly used checksums Body, Fuz1, and Fuz2. Rej-thold and log-thold must be numbers, the string NEVER, or the string MANY indicating millions of targets. Counts from the DCC server as large as the threshold for any single type are taken as sufficient evidence that the message should be logged or rejected.

    Log-thold is the threshold at which messages are logged. It can be handy to log messages at a lower threshold to find solicited bulk mail sources such as mailing lists. If no logging threshold is set, only rejected mail and messages with complicated combinations of white and blacklisting are logged. Messages that reach at least one of their rejection thresholds are logged regardless of logging thresholds.

    Rej-thold is the threshold at which messages are considered "bulk," and so should be rejected or discarded if not whitelisted.

    DCC Reputation thresholds in the commercial version of DCC are controlled by thresholds on checksum types rep and rep-total. The DCC Reputations of IP addresses that the DCC database says have sent more than rep-total,log-thold are computed and messages from those addresses are logged. Messages from IP addresses with DCC Reputations of at least the rep,rej-thold rejection threshold can be rejected. The DCC Reputation of an IP address is the percentage of its messages known to have been sent to at least 10 recipients. The defaults are equivalent to rep,never and rep-total,never,20.

    Bulk DCC Reputations do not reject mail unless enabled by an option DCC-rep-on line in a whiteclnt file.

    The checksums of locally whitelisted messages are not checked with the DCC server and so only the number of targets of the current copy of a whitelisted message are compared against the thresholds.

    The default is ALL,NEVER, so that nothing is discarded, rejected, or logged. A common choice is CMN,25,50 to reject or discard mail with common bodies except as overridden by the whitelist of the DCC server, the sendmail ${dcc_isspam} and ${dcc_notspam} macros, and -g, and -w.

-g  [not-] type indicates that whitelisted, OK or OK2, counts from the DCC server for a type of checksum are to be believed. They should be ignored if prefixed with not-. Type is one of the same set of strings as for -t. Only IP, env_From, and From are likely choices. By default all three are honored, and hence the need for not-.

-S hdr
    adds to the list of substitute or locally chosen headers that are checked with the -w whiteclnt file and sent to the DCC server. The checksum of the last header of type hdr found in the message is checked. Hdr can be HELO to specify the SMTP envelope HELO value. Hdr can also be mail_host to specify the sendmail "resolved" host name from the Mail_from value in the SMTP envelope. As many as six different substitute headers can be specified, but only the checksum of the first of the six will be sent to the DCC server.

-l logdir
    specifies a directory in which files containing copies of messages processed by dccm are kept. They can be copied to per-user directories specified with -U. Information about other recipients of a message is deleted from the per-user copies.

    See the FILES section below concerning the contents of the files. See also the option log-subdirectory-{day,hour,minute} lines in whiteclnt files described in dcc(8).

    The directory is relative to the DCC home directory if it is not absolute

-R rundir
    specifies the "run" directory where the UNIX domain socket and file containing the daemon's process ID are stored. The default value is /var/run/dcc .

-r rejection-msg
    specifies the rejection message in -o proxy mode for unsolicited bulk mail or for mail temporarily blocked by greylisting when -G is specified. The first -r rejection-msg replaces the default bulk mail rejection message, "5.7.1 550 mail %ID from %CIP rejected by DCC". The second replaces "4.2.1 452 mail %ID from %CIP temporary greylist embargoed". The third -r rejection-msg replaces the default SMTP rejection message "5.7.1 550 %ID bad reputation; see http://commercial-dcc.rhyolite.com/cgi-bin/reps.cgi?tgt=%CIP" for mail with bulk DCC Reputations. If rejection-msg is the zero-length string, the -r setting is counted but the corresponding message is not changed.

    Rejection-msg can contain specific information about the mail message. The following strings starting with % are replaced with the corresponding values:
    %ID message ID such as the unique part of log file name or sendmail queue ID
    %CIP
        SMTP client IP address
    %BTYPE
        type of DNS blacklist hit, such as "SMTP client", "mail_host", or "URL NS"
    %BTGT
        IP address or name declared bad by DNS blacklist
    %BPROBE
        domain name found in DNS blacklist such as 4.3.2.10.example.com
    %BRESULT
        value of the %BPROBE domain name found in DNS blacklist

    A common alternate for the bulk mail rejection message is "4.7.1 451 Access denied by DCC" to tell the sending mail system to continue trying. Use a 4yz response with caution, because it is likely to delay for days a delivery failure message for false positives. If the rejection message does not start with an RFC 1893 status code and RFC 2821 reply code, 5.7.1 and 550 or 4.2.1 and 452 are used.

    See also -B set:rej-msg=rejection-msg to set the status message for mail rejected by DNS blacklists.

-j maxjobs
    limits the number of simultaneous requests that will be processed. The default value is the maximum number that seems to be possible given system limits on open files, select() bit masks, and so forth. Start dccm with -d and see the starting message in the system log to see the limit.

-B dnsbl-option
    enables DNS blacklist checks of the SMTP client IP address, SMTP envelope Mail_From sender domain name, and of host names in URLs in the message body. Body URL blacklisting has too many false positives to use on abuse mailboxes. It is less effective than greylisting with dccm(8) or dccifd(8) but can be useful in situations where greylisting cannot be used.

    Dnsbl-option is either one of the -B set:option forms or
        -B
        domain[any[,bltype]]

        -B
        domain[,IPaddr
        [/xx[,bltype]]]

        -B
        domain[,IPaddr
        [-IPaddrHI[,bltype]]]

    Domain is a DNS blacklist domain such as example.com that will be searched. The string any, IPaddr, IPaddr/xxx, or IPaddrLO-IPaddrHI, specifies which IP addresses found in the DNS blacklist say that mail messages should be rejected "127.0.0.2" is assumed if no address(es) are specified. IPv6 addresses can be specified with the usual colon (:) notation. Host names can be used instead of numeric addresses. The type of DNS blacklist is specified by bltype as name, all-names, IPv4, or IPv6. Given an envelope sender domain name or a domain name in a URL of spam.domain.org and a blacklist of type name, spam.domain.org.example.com will be looked up. The names spam.domain.org.example.com, domain.org.example.com, and org.example.com will be looked up in blacklists of type all-names. Use name with DNS blacklists that use wild cards for speed but all-names for other DNS name blacklists. Blacklist types of IPv4 and IPv6 require that the domain name in a URL sender address be resolved into an IPv4 or IPv6 address. The resolved address from the mail message is then written as a reversed string of decimal octets to check the DNS blacklist, as in 2.0.0.127.example.com.

    More than one blacklist can be specified and blacklists can be grouped with -B set:group=X. All searching within a group of blacklists is stopped at the first positive result.

    Positive results are ignored after being logged unless an option DNSBL-on line appears in the global or per-user whiteclnt file.

    -B set:no-client
        implies that SMTP client IP addresses and reverse DNS domain names should not be checked in the following blacklists.
        -B set:client restores the default for the following blacklists.

    -B set:no-mail_host
        implies that SMTP envelope Mail_From sender domain names should not be checked in the following blacklists. -B set:mail_host restores the default.

    -B set:no-URL
        says that URLs in the message body should not be checked in the in the following blacklists. -B set:URL restores the default.

    -B set:no-MX
        says MX servers of sender Mail_From domain names and host names in URLs should not be checked in the following blacklists.
        -B set:MX restores the default.

    -B set:no-NS
        says DNS servers of sender Mail_From domain names and host names in URLs should not be checked in the following blacklists. -B set:NS restores the default.

    -B set:defaults
        is equivalent to all of -B set:client
        -B set:mail_host -B set:URL -B set:MX and -B set:NS

    -B set:group=X
        adds following DNS blacklists specified with
            -B
            domain[,IPaddr
            [/xx[,bltype]]]

        to group 1, 2, or 3.

    -B set:debug=X
        sets the DNS blacklist logging level

    -B set:msg-secs=S
        limits dccm to S seconds total for checking all DNS blacklists. The default is 25.

    -B set:URL-secs=S
        limits dccm to at most S seconds resolving and checking any single URL. The default is 11. Some spam contains dozens of URLs and some "spamvertised" URLs contain host names that need minutes to resolve. Busy mail systems cannot afford to spend minutes checking each incoming mail message.

    -B set:rej-msg=rejection-msg
        sets the SMTP rejection message for the following blacklists. Rejection-msg must be in the same format as for -r. If rejection-msg is null, the default is restored. The default DNS blacklist rejection message is the first message set with -r.

    -B set:maxjobs=X
        sets maximum number of helper processes to X. In order to use typical single-threaded DNS resolver libraries, dccm uses fleets of helper processes. It is rarely a good idea to change the default, which is the same as the maximum number of simultaneous jobs set with -j.

    -B set:progpath=/usr/local/dcc/libexec/dns-helper
        changes the path to the helper program.

-L ltype,facility.level
    specifies how messages should be logged. Ltype must be error, info, or off to indicate which of the two types of messages are being controlled or to turn off all syslog(3) messages from dccm. Level must be a syslog(3) level among EMERG, ALERT, CRIT, ERR, WARNING, NOTICE, INFO, and DEBUG. Facility must be among AUTH, AUTHPRIV, CRON, DAEMON, FTP, KERN, LPR, MAIL, NEWS, USER, UUCP, and LOCAL0 through LOCAL7. The default is equivalent to
        -L info,MAIL.NOTICE -L error,MAIL.ERR

dccm normally sends counts of mail rejected and so forth the to system log at midnight. The SIGUSR1 signal sends an immediate report to the system log. They will be repeated every 24 hours instead of at midnight.

SENDMAIL MACROS
Sendmail can affect dccm with the values of some sendmail.cf macros. These macro names must be added to the Milter.macros option statements in sendmail.cf as in the example "Feature" file dcc.m4.

${dcc_isspam}
    causes a mail message to be reported to the DCC server as having been addressed to "MANY" recipients. The ${dcc_isspam} macro is ignored if the ${dcc_notspam} macro is set to a non-null string

    If the value of the ${dcc_isspam} is null, dccm uses SMTP rejection messages controlled by -a and -r. If the value of the ${dcc_isspam} macro starts with "DISCARD", the mail message is silently discarded as with -a DISCARD. If value of the macro not null and does not start with "DISCARD", it is used as the SMTP error message given to the SMTP client trying to send the rejected message. The message starts with an optional SMTP error type and number followed by text.

    The -a option does not effect messages marked spam with ${dcc_isspam}. When the ${dcc_isspam} macro is set, the message is rejected or discarded despite local or DCC database whitelist entries. The local whitelist does control whether the message's checksums will be reported to the DCC server and an X-DCC SMTP header line will be added.

${dcc_notspam}
    causes a message not be considered unsolicited bulk despite evidence to the contrary. It also prevents dccm from reporting the checksums of the message to the DCC server and from adding an X-DCC header line.

    When the macro is set by the sendmail.cf rules, ${dcc_notspam} macros overrides DCC threshlds that say the message should be rejected as well as the effects of the ${dcc_isspam} macro.

${dcc_mail_host}
    specifies the name of the SMTP client that is sending the message. This macro is usually the same as the mail_host macro. They can differ when a sendmail "smart relay" is involved. The ${dcc_mail_host} macro does not work if FEATURE(delay_checks) is used.

${dcc_userdir}
    is the per-user whitelist and log directory for a recipient. If the macro is not set in sendmail.cf, $&{rcpt_mailer}/$&{rcpt_addr} is assumed, but with the recipient address converted to lower case. Whatever value is used, the directory name after the last slash (/) character is converted to lower case. Any value containing the string "/../" is ignored.

    This macro also does not work if FEATURE(delay_checks) is used.

    The following two lines in a sendmail mc file have the same effect as not defining the ${dcc_userdir} macro, provided FEATURE(dcc) is also used and the sendmail cf/feature directory has a symbolic link to the misc/dcc.m4 file.

    SLocal_check_rcpt
    R$*	$: $1 $(macro {dcc_userdir} $@ $&{rcpt_mailer}/$&{rcpt_addr} $))

FILES
/usr/local/dcc
    is the DCC home directory in which other files are found.
/usr/local/dcc/libexec/start-dccm
    is a script used to dccm.
/usr/local/dcc/dcc_conf
    contains parameters used by the scripts to start DCC daemons and cron jobs.
logdir
    is an optional directory specified with -l and containing marked mail. Each file in the directory contains one message, at least one of whose checksums reached its -t thresholds or that is interesting for some other reason. Each file starts with lines containing the date when the message was received, the IP address of the SMTP client, and SMTP envelope values. Those lines are followed by the body of the SMTP message including its header as it was received by sendmail and without any new or changed header lines. Only approximately the first 32 KBytes of the body are recorded unless modified by ./configure --with-max-log-size=xx The checksums for the message follow the body. They are followed by lines indicating that the ${dcc_isspam} or ${dcc_notspam} sendmail.cf macros were set or one of the checksums is white- or blacklisted by the -w whiteclnt file. Each file ends with the X-DCC header line added to the message and the disposition of the message including SMTP status message if appropriate.
map is the memory mapped file of information concerning DCC servers in the DCC home directory.
whiteclnt
    contains the client whitelist in the format described in dcc(8).
whiteclnt.dccw
    is a memory mapped hash table of the whiteclnt file.
dccm.pid
    in the -R rundir directory contains daemon's process ID. The string “dccm” is replaced by the file name containing the daemon to facilitate running multiple daemons, probably connected to remote instances of sendmail using TCP/IP instead of a UNIX domain socket. See also -R.
/var/run/dcc/dccm
    is the default UNIX domain socket used by the sendmail milter interface. See also -R.
sendmail.cf
    is the sendmail(8) control file.
misc/dcc.m4
    sendmail mc file that should have a symbolic link in the sendmail cf/feature directory so that FEATURE(dcc) can be used in a sendmail mc file.

EXAMPLES
Dccm should be started before sendmail with something like the script /usr/local/dcc/libexec/start-dccm. It looks for common DCC parameters in the /usr/local/dcc/dcc_conf file.

Those numbers should modified to fit local conditions. It might be wise to replace the "100" numbers with much larger values or with "MANY" until a few weeks of monitoring the log directory show that sources of mailing lists are in the server's whitelist file (see dccd(8)) or the local whiteclnt file.

It is usually necessary to regularly delete old log files with a script like /usr/local/dcc/libexec/cron-dccd.

On systems unlike modern FreeBSD and other UNIX-like systems which include sendmail milter support, sendmail must be built with the milter interface, such as by creating a devtools/Site/site.config.m4 or similar file containing something like the following lines:

    APPENDDEF(`conf_sendmail_ENVDEF', `-D_FFR_MILTER=1')
    APPENDDEF(`conf_libmilter_ENVDEF', `-D_FFR_MILTER=1')

Appropriate lines invoking the milter interface must be added to sendmail.cf. That can be done by putting a symbolic link to the the misc/dcc.m4 file in the DCC source to the sendmail cf/feature directory and adding the line

    FEATURE(dcc)

to the local .mc file.

Note that dccm should not be used with the Postfix milter mechanism. Instead use dccifd(8) as a before-queue filter as described in that man page.

SEE ALSO
cdcc(8), dbclean(8), dcc(8), dccd(8), dblist(8), dccifd(8), dccproc(8), dccsight(8), sendmail(8).

HISTORY
Distributed Checksum Clearinghouses are based on an idea of Paul Vixie. Implementation of dccm was started at Rhyolite Software in 2000. This document describes version 1.3.116.

BUGS
dccm uses -t where dccproc(8) uses -c.

Systems without setrlimit(2) and getrlimit(2) RLIMIT_NOFILE can have problems with the default limit on the number of simultaneous jobs, the value of -j. Every job requires four open files. These problems are usually seen with errors messages that say something like
    dccm[24448]: DCC: accept() returned invalid socket
A fix is to use a smaller value for -j or to allow dccm to open more files. Sendmail version 8.13 and later can be told to poll() instead of select with SM_CONF_POLL. Some older versions of sendmail knew about FFR_USE_POLL. One of the following lines in your devtools/Site/site.config.m4 file can help:

    APPENDDEF(`conf_libmilter_ENVDEF', `-DSM_CONF_POLL')
    APPENDDEF(`conf_libmilter_ENVDEF', `-DFFR_USE_POLL')

On many systems with sendmail 8.11.3 and preceding, a bug in the sendmail milter mechanism causes dccm to die with a core file when given a signal.