Linux, Unix, /etc

Danger Will Robinson! You are now entering a condescending Unix user zone!

Procmail: A One-size-fits-all Mail Processor

Introduction

This article shows Unix system administrators how they can enhance their e-mail setup by installing procmail, a general-purpose mail processing program for all brands of Unix. Why procmail? Well, that's really a question this whole article is devoted to answering. But, in essence, you should consider using it because it is a drop-in replacement for /bin/mail that does more, much more. Some other appealing features are its simplicity — just one executable and one config file — and its performance. Written in C, it has a low impact on system resources — a note accompanying the source distribution claims "1.4 times faster than the average /bin/mail in user-cpu time"

Getting

The system requirements are basic:

Any Unix-alike (or POSIX compliant) system;

A compatible MTA (mail transport agent). Basically, any mailer that can process RFC822-compliant mail — Sendmail, Qmail, Postfix, or whatever.

The latest version of the procmail sources is available from the Procmail ftp site. As of writing, the latest version is 3.14, but owing to some minor bugs, pending the arrival of 3.14.1 the latest sable version should be regarded as 3.13.1.

Configuration is a matter of reviewing, and if necessary making changes to, two files, Makefile and config.h, then running make install. These files should not need to be changed, as the make process includes a sophisticated autoconfiguration routine. If not compiling as root, you will have to separate the steps, with make, followed by su and make install.

It is recommended that procmail be installed suid root. You should note the following:

When in explicit delivery mode e.g. when called from sendmail, procmail will always change uid and gid to the recipient's defaults as soon as it starts reading their .procmailrc file.

When not in explicit delivery mode, e.g. when called from the recipient's .forward file, procmail will always change uid and gid to the real uid and gid of the invoker, thus losing any suid and sgid privileges.

These two precautions should effectively eliminate any security holes, since procmail will always have the uid of the user whose commands it is executing.

Using procmail with sendmail

There are a number of ways procmail can be used with sendmail. It can be used as a local mail delivery agent, or as a general-purpose prog mailer. If you are using a recent version of sendmail, you can use the m4 macro files to greatly simplify sendmail configuration for whichever use of procmail you desire. In this case, there are four variables and two features that relate to procmail. The features are:

procmail
local_procmail

and the variables are (with default value in brackets):

PROCMAIL_MAILER_PATH	[/usr/local/bin/procmail] The path to the procmail
			program.  This is also used by FEATURE(local_procmail).
PROCMAIL_MAILER_FLAGS	[SPhnu9] Flags added to Procmail mailer.  Flags
			DFM are always set.  This is NOT used by
			FEATURE(local_procmail); tweak LOCAL_MAILER_FLAGS
			instead.
PROCMAIL_MAILER_ARGS	[procmail -Y -m $h $f $u] The arguments passed to
			the Procmail mailer.  This is NOT used by
			FEATURE(local_procmail); tweak LOCAL_MAILER_ARGS
			instead.
PROCMAIL_MAILER_MAX	[undefined] If set, the maximum size message that
			will be accepted by the procmail mailer.

Local mailer

The simplest way in which procmail can be used is as a drop-in replacement for the local mailer. This entails mail delivery as normal, plus extra processing if a .procmailrc file is present in the home directory of the recipient. To do this, follow these steps:

If you are building a sendmail config file from scratch, or you maintain your sendmail config using the m4 macros, then simply define local_procmail in your .mc file, like so:

FEATURE(local_procmail)

and rebuild sendmail.cf. Note that this does not use PROCMAIL_MAILER_FLAGS or PROCMAIL_MAILER_ARGS for the local mailer; you have tweak LOCAL_MAILER_FLAGS and LOCAL_MAILER_ARGS instead.

If you maintain sendmail.cf directly, then edit the local mailer definition. If you're using a sendmail 8.6.x or older, use this:

Mlocal, P=/usr/local/bin/procmail, F=lsSDFMhPfn, S=10, R=20,
	A=procmail -Y -a $h -d $u

If you're using sendmail 8.7 or newer, then use this: eg( Mlocal, P=/usr/bin/procmail, F=SAw5:|/@glDFMPhsfn, S=10/30, R=20/40, T=DNS/RFC822/X-Unix, A=procmail -Y -a $h -d $u ) Procmail as a local mailer needs root privileges on startup. Further, on some systems, procmail may need to be sgid daemon or mail. Check this by looking at the current mail delivery agent, and copying its permissions. You can also use "make recommend", which will suggest appropriate permissions for your system. The optional lockfile program may also need the same group permissions, to allow it to create and unlink lockfiles in the mail spool directory.

Changing the mail spool directory to the user's home directory You may wish to store mail in the recipient's home directory, rather than in the traditional /usr/spool/mail. This has some obvious advantages:

Mail is subject to the user's quota limitations;

There is often more room on the home partition(s) than on /usr/mail.

The quota limitations also apply to /usr/spool/mail or /usr/mail if procmail does the delivery. These quota limitations often do not work with the regular /bin/mail since that usually writes the mailbox with root permissions (eluding the quota restrictions).

If you intend to install procmail as the local mailer, and you want mail to be delivered to, say, $HOME/.mail by default, then:

1) Uncomment, and if necessary change, SYSTEM_MBOX in config.h, and rebuild the program. Procmail will now deliver to this location by default.

2) All users must have their MAIL environment variable set to this value. If a mail client in use at your site does not use MAIL, then either fix the source of that program, or create symbolic links in /var/mail.

prog mailer

Procmail can also be used as what in sendmail terms is referred to as a prog mailer. This can then be used a general mail filter. Use the procmail feature if configuring sendmail using the m4 macros. The entry in sendmail.cf reads as follows:

Mprocmail, 	P=/usr/local//bin/procmail, F=mSDFMhun, S=11, R=21,
		A=procmail -m $h $g $u

Mailertables

Sendmail and procmail in combination provide a simple way to forward all mail for a given domain to a single person, using the sendmail mailertable feature. This involves defining procmail as a prog mailer. mailertable: eg( example.com procmail:/etc/procmailrcs/example.com ) /etc/procmailrcs/example.com:

:0	# forward mail for example.com
! -oi -f $1 person@other.host

This would send any mail for (anyone)@example.com to to person@other.host. Within the procmail script, $1 is the name of the sender and $2 is the name of the recipient. Note that if you use this with FEATURE(local_procmail), list FEATURE first.

Basic Recipes

The rcfile, whether the global /etc/procmailrc or each user's $HOME/.procmailrc, can have two types of entry: environment variable assignments, and recipes.

The key concept with procmail is the idea of a "recipe". This is a rule in the procmail rc file (/etc/procmailrc or $HOME/.procmailrc). Each recipe has three parts: header, pattern, action. The header identifies this as a recipe, and declares options. The pattern, as the name implies, matches a regular expression against the message header, or body, or both. The action does something if the regular expression matches.

:0 [flags] [ : [locallockfile] ]
[zero or more conditions (one per line)]
[exactly one action line]

A second colon, after the 0 on the first line, tells procmail to use locking. Locking is important; you may have two procmail processes running at the same time, both trying to deliver mail to the same location. A second colon alone says to [?what!?]; followed by a filename, to use that file as the lock file. The lockfile will be created, and unlinked after processing is complete. Thus, if a second instance of procmail should happen along in the meantime, it will see that the file already exists, and wait.

Flags can be any combination of the following (though of course some combinations will not make much sense):

H match the pattern against the header (default)
B match the pattern against the body
D distinguish case in pattern matches (default is ignore case)
A on a match, only follow this recipe if the last recipe without the A flag matched as well
a as A, and additionally the last recipe must have successfully completed
E The inverse of A, that is only follow this recipe if the immediately preceding recipe was not executed.
e Only follow this recipe if the immediately preceding recipe failed— that is, the action line was executed and returned an error code.
h Feed the header to the action line (default).
b Feed the body to the action line (default).
f Treat the action line pipe as a filter.
c Generate a carbon copy of this mail. Processing of the mail continues.
w Wait for the action line filter or program to finish, and check its exit-code (usually ignored). IF the filter is unsuccessful, then the mail is not considered to have been processed.
W As w, suppressing any "program failure" message.
i Ignore any write errors, as one might receive from an early closed pipe, for instance.
r Raw mode: just write the mail out as is. Without this, procmail will make sure that the message ends with an empty line.

The condition mostly uses standard egrep regexp syntax, with a few additions. These are:

! invert the condition.
$ evaluate what follows using the sh(1) substitution rules for matter inside double quotes, skip leading whitespace, then re-parse it.
? Use the exit-code of the specified program.
< Check whether the total length of the message is less than the specified number of bytes.
> Check whether the total length of the message is greater than the specified number of bytes.
variable ?? match the remainder of the condition against the value of this environment variable.
\ Escape, to quote any of the special characters above at the start of a line.

Finally, the action line offers these possibilities:

! Forwards the message to the specified e-mail addresses.
| Starts the specified program. If the command given contains shell metacharacters, a shell will be started to run it in. Variable assignment before this symbol will assign the output of the program to that variable. A pipe symbol alone, with no program, will send the e-mail to stdout.
{ (followed by whitespace) marks the start of a nesting block. Everything until a closing } will depend on the conditions specified for this recipe.

For example:

:0
*
mail

This is a very simple recipe that stores mail in a file called mail. As you can see, the first line of a recipe begins with :, and the second line with *. A recipe can have more than one pattern line, but only one header line and only one action line. Don't confuse the star at the beginning of the pattern line with a wildcard, by the way. The default matching action is "match", so since we haven't got any pattern, this recipe will always succeed.

When writing procmail recipes, it is best to test them first. You can do this easily on the command line. Simply invoke procmail, giving the name of the rc file you wish to test as the first argument, and redirect a sample e-mail message to be procmail's standard input, like so:

$ procmail ./test.rc < ./sample.msg

Where sample.msg is of the form

From: somebody@example.com
To: me@example.com

This is a test

Note the blank line between header and body. Replace the To: address with your own.

Let's test our example recipe.

$ procmail test.rc < test.msg
procmail: [991] Mon Mar  1 17:05:45 1999
procmail: Match on ".*"
procmail: Assigning "LASTFOLDER=mail"
procmail: Opening "mail"
procmail: Acquiring kernel-lock
Folder: mail
$ cat mail
From: somebody@example.com
To: paul

This is a test

Procmail uses special environment variables, which you can change in the rcfile. Here is a list of the more important ones.

LOGABSTRACT if set, procmail writes a one-line summary to the logfile, consisting of the From and Subject fields, where the message went, and its size in bytes.
LOGFILE Where to send error, diagnostic and LOGABSTRACT messages.
SHELL what shell to use — default /bin/sh
VERBOSE turns on extended diagnostics — default off.

Recipes in Action

Here is a simple /etc/procmailrc as an example. This does little more than log all arriving mail in what may be a more convenient and easily-customisable format than sendmail, do some simple MIME conversion (recipe for this taken from procamilex(5)), and implement a very simple "auto-responder", to reply with information to people who write to certain addresses. This is easily changed so that, for example, people writing to abuse get a formula message about how concerned you are about spam ;-), with their message then being sent on to the appropriate person at your site. Normally, we don't want to do much to people's mail at a global level, anyway.

# set some environment variables
SHELL=/bin/sh
LOGFILE=/var/log/procmail
LOGABSTRACT=all
VERBOSE=off

# convert MIME types
:0
^Content-Type: *text/plain
{
  :0 fbw
  * ^Content-Transfer-Encoding: *quoted-printable
  | $MIMENCODE -u -q
 
     :0 Afhw
     | formail -I "Content-Transfer-Encoding: 8bit"
 
  :0 fbw    
  * ^Content-Transfer-Encoding: *base64
  | $MIMENCODE -u -b
 
     :0 Afhw   
     | formail -I "Content-Transfer-Encoding: 8bit"
}

# auto-responder
:0 H
$ ! ^$MYXLOOP 
! ^FROM_DAEMON 
^Subject: *(get|send|mail|gimme)\/( +(info|help))+ 
{

	GETTING=$MATCH		# What got me here?
	MYFROM="From: example.com's Magic Mail Responder "
	SUBJECT=`formail -xSubject`
	BOUNDARY=`formail -x'Message-Id:' | sed 's/[@ ]//g' | cut -c1-65`

	# log this in its own logfile
	LOGFILE=
	AUTOREPLYLOG=/var/log/autoreply-log

  :0 ch:
  $AUTOREPLYLOG

	# This is how I get real headers.
	:0 fhW
	| formail -rY -I"$MYFROM" \
		-I"$MYXLOOP" 

	BOUNDARY=`echo '' ; echo '' ; echo —$BOUNDARY`

	:0 fbW
	| echo '' ; \
	echo $BOUNDARY ;\
	echo '' ;\
	echo 'You asked for it

Resources

Procmail comes with a good set of man pages, which are also available on the WWW:

There is a procmail mailing list. To subscribe, send mail with a Subject line of subscribe to procmail-request@informatik.rwth-aachen.de for the full list, or, for the digest version, to procmail-d-request@informatik.rwth-aachen.de . Some other WWW resources include:

Conclusion

In this article, we've looked at how procmail can enhance local mail handling on a Unix server. There is much more about procmail that we haven't covered, including use of its companion programs formail(1) and lockfile(1). The resources given above should be a help in exploring procmail in more detail. Paul Dunne 2000

[back to Linux, Unix, /etc]