EXIM'S INTERFACES TO MAIL FILTERING Exim is a mail transfer agent for Unix-like systems. This document describes the user interfaces to its in-built mail filtering facilities, and is copyright (c) University of Cambridge 2003. It corresponds to Exim version 4.30. ___________________________________________________________________________ CONTENTS 1. Forwarding and filtering in Exim 2 1.1 Introduction 2 1.2 Testing a new filter file 3 1.3 Installing a filter file 4 1.4 Testing an installed filter file 4 1.5 Details of filtering commands 5 2. Sieve filter files 6 2.1 Recognition of Sieve filters 6 2.2 Saving to specified folders 6 2.3 Strings containing header names 6 2.4 Exists test with empty list of headers 6 2.5 Header test with invalid MIME encoding in header 7 2.6 Address test for multiple addresses per header 7 2.7 Semantics of keep 7 2.8 Semantics of fileinto 7 2.9 Semantics of redirect 7 2.10 String arguments 8 2.11 Number units 8 2.12 RFC compliance 8 3. Exim filter files 10 3.1 Format of Exim filter files 10 3.2 Data values in filter commands 10 3.3 String expansion 11 3.4 Some useful general variables 12 3.5 Header variables 13 3.6 User variables 14 3.7 Current directory 14 3.8 Significant deliveries 14 3.9 Filter commands 15 3.10 The add command 15 3.11 The deliver command 15 3.12 The save command 16 3.13 The pipe command 17 3.14 Mail commands 19 3.15 Logging commands 21 3.16 The finish command 22 3.17 The testprint command 22 3.18 The fail command 22 3.19 The freeze command 22 3.20 The headers command 23 3.21 Obeying commands conditionally 23 3.22 String testing conditions 23 3.23 Numeric testing conditions 25 3.24 Testing for significant deliveries 25 3.25 Testing for error messages 26 3.26 Testing for personal mail 26 3.27 Testing delivery status 27 3.28 Testing a list of addresses 27 3.29 Multiple personal mailboxes 28 3.30 Ignoring delivery errors 29 3.31 Examples of Exim filter commands 29 1. FORWARDING AND FILTERING IN EXIM 1.1 Introduction Most Unix mail transfer agents (programs that deliver mail) permit individ- ual users to specify automatic forwarding of their mail, usually by placing a list of forwarding addresses in a file called .forward in their home directories. Exim extends this facility by allowing the forwarding instruc- tions to be a set of rules rather than just a list of addresses, in effect providing '.forward with conditions'. Operating the set of rules is called filtering, and the file that contains them is called a filter file. Exim supports two different kinds of filter file. An "Exim filter" contains | instructions in a format that is unique to Exim. A "Sieve filter" contains | instructions in the Sieve format that is defined by RFC 3028. As this is a | standard format, Sieve filter files may already be familiar to some users. | Sieve files should also be portable between different environments. | However, the Exim filtering facility contains more features (such as | variable expansion), and better integration with the host environment (such | as the use of external processes and pipes). | | The choice of which kind of filter to use can be left to the end-user, | provided that the system administrator has configured Exim appropriately | for both kinds of filter. However, if interoperability is important, Sieve | is the only choice. | | The ability to use filtering or traditional forwarding has to be enabled by | the system administrator, and some of the individual facilities can be | separately enabled or disabled. A local document should be provided to | describe exactly what has been enabled. In the absence of this, consult | your system administrator. | | It is important to realize that, in Exim, no deliveries are actually made | while a filter or traditional .forward file is being processed. The result | of such processing is a list of destinations to which a message should be | delivered - the deliveries themselves take place later, along with all | other deliveries for the message. This means that it is not possible to | test for successful deliveries while filtering. It also means that any | duplicate addresses that are generated are dropped, since Exim never | delivers the same message to the same address more than once. | | This document describes how to use a filter file and the format of its | contents. It is intended for use by end-users. Both Sieve filters and Exim | filters are covered. However, for Sieve filters, only issues that relate to | the Exim implementation are discussed, since Sieve itself is described | elsewhere. | | The contents of traditional .forward files are not described here. They | normally contain just a list of addresses, file names, or pipe commands, | separated by commas or newlines, but other types of item are also | available. The full details can be found in the chapter on the redirect | router in the Exim specification, which also describes how the system | administrator can set up and control the use of filtering. | 1.2 Testing a new filter file Filter files, especially the more complicated ones, should always be tested, as it is easy to make mistakes. Exim provides a facility for preliminary testing of a filter file before installing it. This tests the syntax of the file and its basic operation, and can also be used with traditional .forward files. Because a filter can do tests on the content of messages, a test message is required. Suppose you have a new filter file called myfilter and a test message called test-message. Assuming that Exim is installed with the conventional path name /usr/sbin/sendmail (some operating systems use /usr/lib/sendmail), the following command can be used: /usr/sbin/sendmail -bf myfilter to e.g. add 2 to n3 There are 10 user variables of this type, and their values can be obtained by the normal expansion syntax (for example $n3) in other commands. At the start of filtering, these variables all contain zero. Both arguments of the "add" command are expanded before use, making it possible to add variables to each other. Subtraction can be obtained by adding negative numbers. 3.11 The deliver command deliver e.g. deliver "Dr Livingstone " This provides a forwarding operation. The message is sent on to the given address, exactly as happens if the address had appeared in a traditional .forward file. If you want to deliver the message to a number of different addresses, you can use more than one "deliver" command (each one may have only one address). However, duplicate addresses are discarded. To deliver a copy of the message to your normal mailbox, your login name can be given as the address. Once an address has been processed by the filtering mechanism, an identical generated address will not be so pro- cessed again, so doing this does not cause a loop. However, if you have a mail alias, you should not refer to it here. For example, if the mail address "L.Gulliver" is aliased to "lg103" then all references in Gulliver's .forward file should be to "lg103". A reference to the alias will not work for messages that are addressed to that alias, since, like .forward file processing, aliasing is performed only once on an address, in order to avoid looping. Following the new address, an optional second address, preceded by "errors_to" may appear. This changes the address to which delivery errors on the forwarded message will be sent. Instead of going to the message's original sender, they go to this new address. For ordinary users, the only value that is permitted for this address is the user whose filter file is being processed. For example, the user "lg103" whose mailbox is in the domain "lilliput.example" could have a filter file that contains deliver jon@elsewhere.example errors_to lg103@lilliput.example Clearly, using this feature makes sense only in situations where not all messages are being forwarded. In particular, bounce messages must not be forwarded in this way, as this is likely to create a mail loop if something goes wrong. 3.12 The save command save e.g. save $home/mail/bookfolder This causes a copy of the message to be appended to the given file (that is, the file is used as a mail folder). More than one "save" command may appear; each one causes a copy of the message to be written to its argument file, provided they are different (duplicate "save" commands are ignored). If the file name does not start with a / character, the contents of the $home variable are prepended, unless it is empty. In conventional configur- ations, this variable is normally set in a user filter to the user's home directory, but the system administrator may set it to some other path. In some configurations, $home may be unset, in which case a non-absolute path name may be generated. Such configurations convert this to an absolute path when the delivery takes place. In a system filter, $home is never set. The user must of course have permission to write to the file, and the writing of the file takes place in a process that is running as the user, under the user's primary group. Any secondary groups to which the user may belong are not normally taken into account, though the system administrator can configure Exim to set them up. In addition, the ability to use this command at all is controlled by the system administrator - it may be forbidden on some systems. An optional mode value may be given after the file name. The value for the mode is interpreted as an octal number, even if it does not begin with a zero. For example: save /some/folder 640 This makes it possible for users to override the system-wide mode setting for file deliveries, which is normally 600. If an existing file does not have the correct mode, it is changed. An alternative form of delivery may be enabled on your system, in which each message is delivered into a new file in a given directory. If this is the case, this functionality can be requested by giving the directory name terminated by a slash after the "save" command, for example save separated/messages/ There are several different formats for such deliveries; check with your system administrator or local documentation to find out which (if any) are available on your system. If this functionality is not enabled, the use of a path name ending in a slash causes an error. 3.13 The pipe command pipe e.g. pipe "$home/bin/countmail $sender_address" This command sets up delivery to a specified command using a pipe. Remember, however, that no deliveries are done while the filter is being processed. All deliveries happen later on. Therefore, the result of running the pipe is not available to the filter. When the deliveries are done, a separate process is run, and a copy of the message is passed on its standard input. The process runs as the user, under the user's primary group. Any secondary groups to which the user may belong are not normally taken into account, though the system administrator can configure Exim to set them up. More than one "pipe" command may appear; each one causes a copy of the message to be written to its argument pipe, provided they are different (duplicate "pipe" commands are ignored). When the time comes to transport the message, the command supplied to "pipe" is split up by Exim into a command name and a number of arguments. These are delimited by white space except for arguments enclosed in double quotes, in which case backslash is interpreted as an escape, or in single quotes, in which case no escaping is recognized. Note that as the whole command is normally supplied in double quotes, a second level of quoting is required for internal double quotes. For example: pipe "$home/myscript \"size is $message_size\"" String expansion is performed on the separate components after the line has been split up, and the command is then run directly by Exim; it is not run under a shell. Therefore, substitution cannot change the number of argu- ments, nor can quotes, backslashes or other shell metacharacters in variables cause confusion. Documentation for some programs that are normally run via this kind of pipe often suggest that the command should start with IFS=" " This is a shell command, and should not be present in Exim filter files, since it does not normally run the command under a shell. However, there is an option that the administrator can set to cause a shell to be used. In this case, the entire command is expanded as a single string and passed to the shell for interpretation. It is recommended that this be avoided if at all possible, since it can lead to problems when inserted variables contain shell metacharacters. The default PATH set up for the command is determined by the system administrator, usually containing at least "/usr/bin" so that common commands are available without having to specify an absolute file name. However, it is possible for the system administrator to restrict the pipe facility so that the command name must not contain any / characters, and must be found in one of the directories in the configured PATH. It is also possible for the system administrator to lock out the use of the "pipe" command altogether. When the command is run, a number of environment variables are set up. The complete list for pipe deliveries may be found in the Exim reference manual. Those that may be useful for pipe deliveries from user filter files are: DOMAIN the domain of the address HOME your home directory LOCAL_PART see below LOCAL_PART_PREFIX see below LOCAL_PART_SUFFIX see below LOGNAME your login name MESSAGE_ID the message's unique id PATH the command search path RECIPIENT the complete recipient address SENDER the sender of the message SHELL /bin/sh USER see below LOCAL_PART, LOGNAME, and USER are all set to the same value, namely, your login id. LOCAL_PART_PREFIX and LOCAL_PART_SUFFIX may be set if Exim is configured to recognize prefixes or suffixes in the local parts of addresses. For example, a message addressed to "pat-suf2@domain.example" may cause user "pat"'s filter file to be run. If this sets up a pipe delivery, LOCAL_PART_SUFFIX is "-suf2" when the pipe command runs. The system administrator has to configure Exim specially for this feature to be available. If you run a command that is a shell script, be very careful in your use of data from the incoming message in the commands in your script. RFC 2822 is very generous in the characters that are legally permitted to appear in mail addresses, and in particular, an address may begin with a vertical bar or a slash. For this reason you should always use quotes round any arguments that involve data from the message, like this: /some/command '$SENDER' so that inserted shell meta-characters do not cause unwanted effects. Remember that, as was explained earlier, the pipe command is not run at the time the filter file is interpreted. The filter just defines what deliver- ies are required for one particular addressee of a message. The deliveries themselves happen later, once Exim has decided everything that needs to be done for the message. A consequence of this is that you cannot inspect the return code from the pipe command from within the filter. Neverthless, the code returned by the command is important, because Exim uses it to decide whether the delivery has succeeded or failed. The command should return a zero completion code if all has gone well. Most non-zero codes are treated by Exim as indicating a failure of the pipe. This is treated as a delivery failure, causing the message to be returned to its sender. However, there are some completion codes which are treated as temporary errors. The message remains on Exim's spool disk, and the delivery is tried again later, though it will ultimately time out if the delivery failures go on too long. The completion codes to which this applies can be specified by the system administrator; the default values are 73 and 75. The pipe command should not normally write anything to its standard output or standard error file descriptors. If it does, whatever is written is normally returned to the sender of the message as a delivery error, though this action can be varied by the system administrator. 3.14 Mail commands There are two commands which cause the creation of a new mail message, neither of which count as a significant delivery unless the command is preceded by the word "seen". This is a powerful facility, but it should be used with care, because of the danger of creating infinite sequences of messages. The system administrator can forbid the use of these commands altogether. To help prevent runaway message sequences, these commands have no effect when the incoming message is a delivery error message, and messages sent by this means are treated as if they were reporting delivery errors. Thus they should never themselves cause a delivery error message to be returned. The basic mail-sending command is mail [to ] [cc ] [bcc ] [from
] [reply_to
] [subject ] [extra_headers ] | [text ] [[expand] file ] [return message] [log ] [once ] [once_repeat