










          SENDMAIL -- An Internetwork Mail Router

                        Eric Allman*
             _U_n_i_v_e_r_s_i_t_y _o_f _C_a_l_i_f_o_r_n_i_a_, _B_e_r_k_e_l_e_y
                      _M_a_m_m_o_t_h _P_r_o_j_e_c_t




                          ABSTRACT

    Routing  mail  through  a heterogenous internet pre-
    sents many new problems.  Among the worst  of  these
    is  that of address mapping.  Historically, this has
    been handled on an _a_d _h_o_c basis.  However, this  ap-
    proach has become unmanageable as internets grow.

    Sendmail  acts  a unified "post office" to which all
    mail can be submitted.   Address  interpretation  is
    controlled  by  a production system, which can parse
    both domain-based addressing and  old-style  _a_d  _h_o_c
    addresses.  The production system is powerful enough
    to rewrite addresses in the message header  to  con-
    form  to  the standards of a number of common target
    networks, including old  (NCP/RFC733)  Arpanet,  new
    (TCP/RFC822)  Arpanet, UUCP, and Phonenet.  Sendmail
    also implements an SMTP  server,  message  queueing,
    and aliasing.





     _S_e_n_d_m_a_i_l implements a general internetwork mail routing

facility, featuring aliasing and forwarding, automatic rout-

ing to network gateways, and flexible configuration.


     In  a  simple  network,  each  node has an address, and

resources can be identified with a  host-resource  pair;  in
____________________
   *A  considerable  part  of this work was done while under
the employ of the INGRES Project at the University of  Cali-
fornia at Berkeley and at Britton Lee.




SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr              SSMMMM::99--11







SSMMMM::99--22              SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr


particular, the mail system can refer to users using a host-

username pair.  Host names and numbers have to  be  adminis-

tered  by a central authority, but usernames can be assigned

locally to each host.


     In an internet, multiple networks with different  char-

acterstics and managements must communicate.  In particular,

the syntax and semantics of resource identification  change.

Certain  special  cases  can  be handled trivially by _a_d _h_o_c

techniques, such as  providing  network  names  that  appear

local  to  hosts  on other networks, as with the Ethernet at

Xerox PARC.  However,  the general case  is  extremely  com-

plex.   For  example,  some  networks require point-to-point

routing, which simplifies the database update problem  since

only  adjacent hosts must be entered into the system tables,

while others use end-to-end addressing.  Some networks use a

left-associative  syntax  and others use a right-associative

syntax, causing ambiguity in mixed addresses.


     Internet standards seek to  eliminate  these  problems.

Initially,  these  proposed  expanding  the address pairs to

address triples, consisting  of  {network,  host,  resource}

triples.   Network  numbers must be universally agreed upon,

and hosts can be assigned  locally  on  each  network.   The

user-level  presentation  was  quickly  expanded  to address

domains, comprised of a local resource identification and  a

hierarchical domain specification with a common static root.

The domain technique separates the issue of physical  versus










SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr              SSMMMM::99--33


logical  addressing.   For  example,  an address of the form

"eric@a.cc.berkeley.arpa" describes only the logical organi-

zation of the address space.


     _S_e_n_d_m_a_i_l is intended to help bridge the gap between the

totally _a_d _h_o_c world of networks that know nothing  of  each

other and the clean, tightly-coupled world of unique network

numbers.  It can  accept  old  arbitrary  address  syntaxes,

resolving ambiguities using heuristics specified by the sys-

tem administrator, as well as domain-based  addressing.   It

helps  guide  the conversion of message formats between dis-

parate networks.  In short, _s_e_n_d_m_a_i_l is designed to assist a

graceful  transition  to  consistent internetwork addressing

schemes.



     Section 1 discusses  the  design  goals  for  _s_e_n_d_m_a_i_l.

Section  2  gives  an overview of the basic functions of the

system.  In section 3, details of usage are discussed.  Sec-

tion 4 compares _s_e_n_d_m_a_i_l to other internet mail routers, and

an evaluation of _s_e_n_d_m_a_i_l is given in section  5,  including

future plans.


11..  DDEESSIIGGNN GGOOAALLSS


        Design goals for _s_e_n_d_m_a_i_l include:


    (1)   Compatibility  with  the  existing  mail programs,

          including Bell version 6 mail, Bell version 7 mail

          [UNIX83],  Berkeley  _M_a_i_l [Shoens79], BerkNet mail










SSMMMM::99--44              SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr


          [Schmidt79], and hopefully UUCP  mail  [Nowitz78a,

          Nowitz78b].   ARPANET  mail [Crocker77a, Postel77]

          was also required.


    (2)   Reliability, in the  sense  of  guaranteeing  that

          every  message  is correctly delivered or at least

          brought to the attention of a  human  for  correct

          disposal;  no  message  should  ever be completely

          lost.  This goal was considered essential  because

          of  the  emphasis  on mail in our environment.  It

          has turned out to be one of the hardest  goals  to

          satisfy, especially in the face of the many anoma-

          lous message formats produced by  various  ARPANET

          sites.    For   example,  certain  sites  generate

          improperly formated addresses, occasionally  caus-

          ing error-message loops.  Some hosts use blanks in

          names, causing problems with  UNIX  mail  programs

          that  assume  that  an  address  is one word.  The

          semantics of some fields are interpreted  slightly

          differently  by  different sites.  In summary, the

          obscure features  of  the  ARPANET  mail  protocol

          really  _a_r_e used and are difficult to support, but

          must be supported.


    (3)   Existing software to do actual delivery should  be

          used whenever possible.  This goal derives as much

          from political  and  practical  considerations  as

          technical.










SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr              SSMMMM::99--55


    (4)   Easy  expansion  to  fairly  complex environments,

          including multiple connections to a single network

          type  (such  as  with  multiple UUCP or Ether nets

          [Metcalfe76]).  This goal  requires  consideration

          of  the contents of an address as well as its syn-

          tax in order to determine which  gateway  to  use.

          For  example,  the  ARPANET is bringing up the TCP

          protocol to replace the old NCP protocol.  No host

          at Berkeley runs both TCP and NCP, so it is neces-

          sary to look at the ARPANET host name to determine

          whether  to  route mail to an NCP gateway or a TCP

          gateway.


    (5)   Configuration should  not  be  compiled  into  the

          code.  A single compiled program should be able to

          run as is at any site (barring such basic  changes

          as the CPU type or the operating system).  We have

          found this seemingly unimportant goal to be criti-

          cal  in  real  life.   Besides the simple problems

          that occur when any program gets recompiled  in  a

          different environment, many sites like to "fiddle"

          with anything that they will be  recompiling  any-

          way.


    (6)   _S_e_n_d_m_a_i_l  must be able to let various groups main-

          tain their own mailing lists, and let  individuals

          specify  their  own  forwarding, without modifying

          the system alias file.










SSMMMM::99--66              SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr


    (7)   Each user should be able to specify  which  mailer

          to  execute  to  process  mail being delivered for

          him.  This feature allows users who are using spe-

          cialized  mailers  that  use a different format to

          build their environment without changing the  sys-

          tem,  and  facilitates specialized functions (such

          as returning an "I am on vacation" message).


    (8)   Network traffic should be  minimized  by  batching

          addresses to a single host where possible, without

          assistance from the user.


        These goals motivated the  architecture  illustrated

   in  figure  1.  The user interacts with a mail generating


____________________________________________________________

          +---------+   +---------+   +---------+
          | sender1 |   | sender2 |   | sender3 |
          +---------+   +---------+   +---------+
               |       |             |
               +----------+  +  +----------+
                    |  |  |
                    v  v  v
                      +-------------+
                      |   sendmail  |
                      +-------------+
                    |  |  |
               +----------+  +  +----------+
               |       |             |
               v             v             v
          +---------+   +---------+   +---------+
          | mailer1 |   | mailer2 |   | mailer3 |
          +---------+   +---------+   +---------+

           Figure 1 -- Sendmail System Structure.
____________________________________________________________














SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr              SSMMMM::99--77


   and sending program.  When the mail is created, the  gen-

   erator  calls  _s_e_n_d_m_a_i_l,  which routes the message to the

   correct mailer(s).  Since some of the senders may be net-

   work  servers  and  some  of  the  mailers may be network

   clients, _s_e_n_d_m_a_i_l may be used as an internet  mail  gate-

   way.


22..  OOVVEERRVVIIEEWW


   22..11..  SSyysstteemm OOrrggaanniizzaattiioonn


           _S_e_n_d_m_a_i_l  neither  interfaces  with  the user nor

      does actual mail delivery.  Rather, it collects a mes-

      sage  generated by a user interface program (UIP) such

      as Berkeley _M_a_i_l, MS [Crocker77b], or  MH  [Borden79],

      edits  the message as required by the destination net-

      work, and calls appropriate mailers to do mail  deliv-

      ery or queueing for network transmission1.  This  dis-

      cipline allows the insertion of new mailers at minimum

      cost.  In this sense _s_e_n_d_m_a_i_l  resembles  the  Message

      Processing Module (MPM) of [Postel79b].


   22..22..  IInntteerrffaacceess ttoo tthhee OOuuttssiiddee WWoorrlldd


           There  are  three  ways  _s_e_n_d_m_a_i_l can communicate

      with the outside world, both in receiving and in send-

      ing  mail.   These  are  using  the  conventional UNIX
____________________
   1except  when  mailing  to a file, when _s_e_n_d_m_a_i_l does the
delivery directly.












SSMMMM::99--88              SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr


      argument vector/return status, speaking  SMTP  over  a

      pair  of  UNIX pipes, and speaking SMTP over an inter-

      process(or) channel.


      22..22..11..  AArrgguummeenntt vveeccttoorr//eexxiitt ssttaattuuss


              This technique is the standard UNIX method for

         communicating  with the process.  A list of recipi-

         ents is sent in the argument vector, and  the  mes-

         sage  body is sent on the standard input.  Anything

         that the mailer prints is simply collected and sent

         back to the sender if there were any problems.  The

         exit status from the mailer is collected after  the

         message  is  sent,  and  a diagnostic is printed if

         appropriate.


      22..22..22..  SSMMTTPP oovveerr ppiippeess


              The SMTP protocol [Postel82] can  be  used  to

         run  an  interactive  lock-step  interface with the

         mailer.  A subprocess  is  still  created,  but  no

         recipient  addresses  are  passed to the mailer via

         the argument list.  Instead, they are passed one at

         a  time  in commands sent to the processes standard

         input.  Anything appearing on the  standard  output

         must be a reply code in a special format.
















SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr              SSMMMM::99--99


      22..22..33..  SSMMTTPP oovveerr aann IIPPCC ccoonnnneeccttiioonn


              This  technique  is  similar  to  the previous

         technique, except that it uses a 4.2bsd IPC channel

         [UNIX83].  This method is exceptionally flexible in

         that  the  mailer  need  not  reside  on  the  same

         machine.  It is normally used to connect to a send-

         mail process on another machine.


   22..33..  OOppeerraattiioonnaall DDeessccrriippttiioonn


           When a sender wants to send a message, it  issues

      a  request  to _s_e_n_d_m_a_i_l using one of the three methods

      described above.  _S_e_n_d_m_a_i_l operates  in  two  distinct

      phases.   In  the  first phase, it collects and stores

      the message.  In the second  phase,  message  delivery

      occurs.  If there were errors during processing during

      the second phase, _s_e_n_d_m_a_i_l creates and returns  a  new

      message  describing the error and/or returns an status

      code telling what went wrong.


      22..33..11..  AArrgguummeenntt pprroocceessssiinngg aanndd aaddddrreessss ppaarrssiinngg


              If _s_e_n_d_m_a_i_l is called using  one  of  the  two

         subprocess  techniques,  the  arguments  are  first

         scanned and option  specifications  are  processed.

         Recipient addresses are then collected, either from

         the command line or from the SMTP RCPT command, and

         a  list  of  recipients  is  created.   Aliases are











SSMMMM::99--1100             SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr


         expanded at this step, including mailing lists.  As

         much  validation  as  possible  of the addresses is

         done at this step: syntax  is  checked,  and  local

         addresses  are  verified,  but detailed checking of

         host names and addresses is deferred  until  deliv-

         ery.   Forwarding  is  also  performed as the local

         addresses are verified.


              _S_e_n_d_m_a_i_l appends each address to the recipient

         list after parsing.  When a name is aliased or for-

         warded, the old name is retained in the list, and a

         flag is set that tells the delivery phase to ignore

         this recipient.  This list is kept free from dupli-

         cates,  preventing  alias  loops and duplicate mes-

         sages deliverd to  the  same  recipient,  as  might

         occur if a person is in two groups.


      22..33..22..  MMeessssaaggee ccoolllleeccttiioonn


              _S_e_n_d_m_a_i_l  then collects the message.  The mes-

         sage should have a header  at  the  beginning.   No

         formatting  requirements are imposed on the message

         except that they  must  be  lines  of  text  (i.e.,

         binary  data is not allowed).  The header is parsed

         and stored in memory, and the body of  the  message

         is saved in a temporary file.


              To simplify the program interface, the message

         is collected even if no addresses were valid.   The










SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr             SSMMMM::99--1111


         message will be returned with an error.


      22..33..33..  MMeessssaaggee ddeelliivveerryy


              For each unique mailer and host in the recipi-

         ent list, _s_e_n_d_m_a_i_l calls  the  appropriate  mailer.

         Each mailer invocation sends to all users receiving

         the message on one host.  Mailers that only  accept

         one recipient at a time are handled properly.


              The message is sent to the mailer using one of

         the same three interfaces used to submit a  message

         to sendmail.  Each copy of the message is prepended

         by a customized header.  The mailer status code  is

         caught  and  checked,  and a suitable error message

         given as appropriate.  The exit code  must  conform

         to a system standard or a generic message ("Service

         unavailable") is given.


      22..33..44..  QQuueeuueeiinngg ffoorr rreettrraannssmmiissssiioonn


              If the mailer returned an  status  that  indi-

         cated  that  it  might  be  able to handle the mail

         later, _s_e_n_d_m_a_i_l will queue the mail and  try  again

         later.


      22..33..55..  RReettuurrnn ttoo sseennddeerr


              If  errors  occur  during processing, _s_e_n_d_m_a_i_l

         returns   the   message   to   the    sender    for











SSMMMM::99--1122             SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr


         retransmission.   The  letter can be mailed back or

         written in the file "dead.letter" in  the  sender's

         home directory2.


   22..44..  MMeessssaaggee HHeeaaddeerr EEddiittiinngg


           Certain editing  of  the  message  header  occurs

      automatically.   Header  lines  can  be inserted under

      control of the configuration file.  Some lines can  be

      merged; for example, a "From:" line and a "Full-name:"

      line can be merged under certain circumstances.


   22..55..  CCoonnffiigguurraattiioonn FFiillee


           Almost all configuration information is  read  at

      runtime from an ASCII file, encoding macro definitions

      (defining the value of macros used internally), header

      declarations  (telling  sendmail  the format of header

      lines that it will process specially, i.e., lines that

      it  will  add or reformat), mailer definitions (giving

      information such as the location  and  characteristics

      of  each  mailer), and address rewriting rules (a lim-

      ited production system to rewrite addresses  which  is

      used to parse and rewrite the addresses).

____________________
   2Obviously, if the site giving the error is not the orig-
inating site, the only reasonable option is to mail back  to
the sender.  Also, there are many more error disposition op-
tions, but they only effect the error message -- the "return
to  sender"  function  is always handled in one of these two
ways.












SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr             SSMMMM::99--1133


           To  improve performance when reading the configu-

      ration file, a memory image  can  be  provided.   This

      provides  a "compiled" form of the configuration file.


33..  UUSSAAGGEE AANNDD IIMMPPLLEEMMEENNTTAATTIIOONN


   33..11..  AArrgguummeennttss


           Arguments may be flags and addresses.  Flags  set

      various processing options.  Following flag arguments,

      address arguments may be given, unless we are  running

      in  SMTP  mode.  Addresses follow the syntax in RFC822

      [Crocker82] for ARPANET address  formats.   In  brief,

      the format is:


       (1)   Anything  in  parentheses  is thrown away (as a

             comment).


       (2)   Anything in angle brackets ("<>") is  preferred

             over  anything  else.  This rule implements the

             ARPANET standard that addresses of the form


                 user name <machine-address>


             will send to the  electronic  "machine-address"

             rather than the human "user name."


       (3)   Double  quotes ( " ) quote phrases; backslashes

             quote characters.  Backslashes are more  power-

             ful  in  that they will cause otherwise equiva-

             lent phrases  to  compare  differently  --  for










SSMMMM::99--1144             SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr


             example,  _u_s_e_r  and  _"_u_s_e_r_" are equivalent, but

             _\_u_s_e_r is different from either of them.


           Parentheses, angle brackets,  and  double  quotes

      must  be  properly balanced and nested.  The rewriting

      rules control remaining parsing3.


   33..22..  MMaaiill ttoo FFiilleess aanndd PPrrooggrraammss


           Files and programs are legitimate message recipi-

      ents.  Files provide  archival  storage  of  messages,

      useful  for  project administration and history.  Pro-

      grams are useful as recipients in a variety of  situa-

      tions, for example, to maintain a public repository of

      systems messages (such as the Berkeley  _m_s_g_s  program,

      or the MARS system [Sattley78]).


           Any  address  passing through the initial parsing

      algorithm as a local address (i.e, not appearing to be

      a valid address for another mailer) is scanned for two

      special cases.  If prefixed by a  vertical  bar  ("|")

      the  rest  of the address is processed as a shell com-

      mand.  If the user name begins with a slash mark ("/")

      the  name  is  used as a file name, instead of a login

      name.



____________________
   3Disclaimer:  Some  special  processing  is  done   after
rewriting local names; see below.












SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr             SSMMMM::99--1155


           Files that have setuid or setgid bits set but  no

      execute  bits  set have those bits honored if _s_e_n_d_m_a_i_l

      is running as root.


   33..33..  AAlliiaassiinngg,, FFoorrwwaarrddiinngg,, IInncclluussiioonn


           _S_e_n_d_m_a_i_l  reroutes  mail  three  ways.   Aliasing

      applies  system  wide.  Forwarding allows each user to

      reroute  incoming  mail  destined  for  that  account.

      Inclusion  directs  _s_e_n_d_m_a_i_l to read a file for a list

      of addresses, and is normally used in conjunction with

      aliasing.


      33..33..11..  AAlliiaassiinngg


              Aliasing  maps  names to address lists using a

         system-wide file.  This file is  indexed  to  speed

         access.  Only names that parse as local are allowed

         as aliases; this guarantees  a  unique  key  (since

         there are no nicknames for the local host).


      33..33..22..  FFoorrwwaarrddiinngg


              After  aliasing, recipients that are local and

         valid are checked for the existence of a ".forward"

         file  in  their  home directory.  If it exists, the

         message is _n_o_t sent to that user, but rather to the

         list  of  users in that file.  Often this list will

         contain only one address, and the feature  will  be

         used for network mail forwarding.










SSMMMM::99--1166             SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr


              Forwarding  also  permits  a user to specify a

         private incoming mailer.  For  example,  forwarding

         to:


             "|/usr/local/newmail myname"


         will use a different incoming mailer.


      33..33..33..  IInncclluussiioonn


              Inclusion is specified in RFC 733 [Crocker77a]

         syntax:


             :Include: pathname


         An address of this form reads the file specified by

         _p_a_t_h_n_a_m_e  and  sends  to  all  users listed in that

         file.


              The intent is _n_o_t to  support  direct  use  of

         this feature, but rather to use this as a subset of

         aliasing.  For example, an alias of the form:


             project: :include:/usr/project/userlist


         is a method of letting a project maintain a mailing

         list  without  interaction with the system adminis-

         tration, even if the alias file is protected.


              It is not necessary to rebuild  the  index  on

         the   alias  database  when  a  :include:  list  is

         changed.










SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr             SSMMMM::99--1177


   33..44..  MMeessssaaggee CCoolllleeccttiioonn


           Once all recipient addresses are parsed and veri-

      fied,  the message is collected.  The message comes in

      two parts: a message header and a message body,  sepa-

      rated by a blank line.


           The  header  is formatted as a series of lines of

      the form


               field-name: field-value


      Field-value can be split across lines by starting  the

      following  lines  with  a space or a tab.  Some header

      fields have special internal meaning, and have  appro-

      priate  special  processing.  Other headers are simply

      passed through.  Some header fields may be added auto-

      matically, such as time stamps.


           The  body  is a series of text lines.  It is com-

      pletely uninterpreted and untouched, except that lines

      beginning  with a dot have the dot doubled when trans-

      mitted over  an  SMTP  channel.   This  extra  dot  is

      stripped by the receiver.


   33..55..  MMeessssaaggee DDeelliivveerryy


           The  send  queue  is  ordered  by  receiving host

      before transmission  to  implement  message  batching.

      Each address is marked as it is sent so rescanning the











SSMMMM::99--1188             SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr


      list is safe.  An argument list is built as  the  scan

      proceeds.   Mail  to files is detected during the scan

      of the send list.  The interface to the mailer is per-

      formed  using  one of the techniques described in sec-

      tion 2.2.


           After a connection is established, _s_e_n_d_m_a_i_l makes

      the  per-mailer  changes  to  the header and sends the

      result to the mailer.  If any mail is rejected by  the

      mailer,  a  flag is set to invoke the return-to-sender

      function after all delivery completes.


   33..66..  QQuueeuueedd MMeessssaaggeess


           If the mailer returns a "temporary failure"  exit

      status, the message is queued.  A control file is used

      to describe the recipients to be sent to  and  various

      other parameters.  This control file is formatted as a

      series of lines, each describing a sender,  a  recipi-

      ent,  the  time  of  submission, or some other salient

      parameter of the message.  The header of  the  message

      is  stored in the control file, so that the associated

      data file in the queue is just the temporary file that

      was originally collected.


   33..77..  CCoonnffiigguurraattiioonn


           Configuration  is  controlled primarily by a con-

      figuration file read at startup.  _S_e_n_d_m_a_i_l should  not











SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr             SSMMMM::99--1199


      need to be recomplied except


       (1)   To change operating systems (V6, V7/32V, 4BSD).


       (2)   To remove or insert  the  DBM  (UNIX  database)

             library.


       (3)   To change ARPANET reply codes.


       (4)   To  add  headers  fields requiring special pro-

             cessing.


      Adding mailers or changing parsing  (i.e.,  rewriting)

      or routing information does not require recompilation.


           If the mail is being sent by a  local  user,  and

      the  file ".mailcf" exists in the sender's home direc-

      tory, that file is read as a configuration file  after

      the  system  configuration  file.   The primary use of

      this feature is to add header lines.


           The configuration file encodes macro definitions,

      header   definitions,  mailer  definitions,  rewriting

      rules, and options.


      33..77..11..  MMaaccrrooss


              Macros can be used  in  three  ways.   Certain

         macros  transmit  unstructured  textual information

         into the mail system, such  as  the  name  _s_e_n_d_m_a_i_l

         will  use  to  identify  itself  in error messages.











SSMMMM::99--2200             SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr


         Other macros transmit information from _s_e_n_d_m_a_i_l  to

         the  configuration  file  for use in creating other

         fields (such as argument vectors to mailers); e.g.,

         the  name  of  the sender, and the host and user of

         the recipient.  Other macros are unused internally,

         and  can  be used as shorthand in the configuration

         file.


      33..77..22..  HHeeaaddeerr ddeeccllaarraattiioonnss


              Header declarations  inform  _s_e_n_d_m_a_i_l  of  the

         format  of  known header lines.  Knowledge of a few

         header lines is built into _s_e_n_d_m_a_i_l,  such  as  the

         "From:" and "Date:" lines.


              Most  configured headers will be automatically

         inserted in the  outgoing  message  if  they  don't

         exist in the incoming message.  Certain headers are

         suppressed by some mailers.


      33..77..33..  MMaaiilleerr ddeeccllaarraattiioonnss


              Mailer declarations tell _s_e_n_d_m_a_i_l of the vari-

         ous mailers available to it.  The definition speci-

         fies the internal name of the mailer, the  pathname

         of  the program to call, some flags associated with

         the mailer, and an argument vector to  be  used  on

         the call; this vector is macro-expanded before use.













SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr             SSMMMM::99--2211


      33..77..44..  AAddddrreessss rreewwrriittiinngg rruulleess


              The heart of address parsing in _s_e_n_d_m_a_i_l is  a

         set  of rewriting rules.  These are an ordered list

         of pattern-replacement rules, (somewhat like a pro-

         duction  system,  except  that  order is critical),

         which are applied to each address.  The address  is

         rewritten  textually  until  it is either rewritten

         into a special canonical  form  (i.e.,  a  (mailer,

         host,  user)  3-tuple,  such as {arpanet, usc-isif,

         postel}  representing  the   address   "postel@usc-

         isif"),  or  it  falls off the end.  When a pattern

         matches, the rule is reapplied until it fails.


              The configuration file also supports the edit-

         ing of addresses into different formats.  For exam-

         ple, an address of the form:


             ucsfcgl!tef


         might be mapped into:


             tef@ucsfcgl.UUCP


         to conform to the domain syntax.  Translations  can

         also be done in the other direction.


      33..77..55..  OOppttiioonn sseettttiinngg


              There are several options that can be set from

         the  configuration   file.    These   include   the










SSMMMM::99--2222             SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr


         pathnames   of  various  support  files,  timeouts,

         default modes, etc.


44..  CCOOMMPPAARRIISSOONN WWIITTHH OOTTHHEERR MMAAIILLEERRSS


   44..11..  DDeelliivveerrmmaaiill


           _S_e_n_d_m_a_i_l is an  outgrowth  of  _d_e_l_i_v_e_r_m_a_i_l.   The

      primary differences are:


       (1)   Configuration  information  is not compiled in.

             This change simplifies many of the problems  of

             moving  to other machines.  It also allows easy

             debugging of new mailers.


       (2)   Address parsing is more flexible.  For example,

             _d_e_l_i_v_e_r_m_a_i_l  only  supported one gateway to any

             network, whereas _s_e_n_d_m_a_i_l can be  sensitive  to

             host names and reroute to different gateways.


       (3)   Forwarding and :include: features eliminate the

             requirement  that  the  system  alias  file  be

             writable by any user (or that an update program

             be written, or that the  system  administration

             make all changes).


       (4)   _S_e_n_d_m_a_i_l  supports message batching across net-

             works when a message is being sent to  multiple

             recipients.













SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr             SSMMMM::99--2233


       (5)   A  mail  queue  is  provided in _s_e_n_d_m_a_i_l_.  Mail

             that cannot be delivered  immediately  but  can

             potentially  be  delivered  later  is stored in

             this queue for a later retry.  The  queue  also

             provides a buffer against system crashes; after

             the message has been collected it may be  reli-

             ably  redelivered  even  if  the system crashes

             during the initial delivery.


       (6)   _S_e_n_d_m_a_i_l uses the networking  support  provided

             by  4.2BSD  to  provide a direct interface net-

             works such as the ARPANET and/or Ethernet using

             SMTP (the Simple Mail Transfer Protocol) over a

             TCP/IP connection.


   44..22..  MMMMDDFF


           MMDF [Crocker79] spans a wider problem  set  than

      _s_e_n_d_m_a_i_l.   For example, the domain of MMDF includes a

      "phone network" mailer, whereas _s_e_n_d_m_a_i_l calls on pre-

      existing mailers in most cases.


           MMDF  and  _s_e_n_d_m_a_i_l  both  support aliasing, cus-

      tomized mailers, message batching, automatic  forward-

      ing  to  gateways, queueing, and retransmission.  MMDF

      supports two-stage timeout, which  _s_e_n_d_m_a_i_l  does  not

      support.














SSMMMM::99--2244             SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr


           The  configuration  for MMDF is compiled into the

      code4.


           Since MMDF does not consider  backwards  compati-

      bility  as  a design goal, the address parsing is sim-

      pler but much less flexible.


           It is somewhat harder to integrate a new channel5

      into MMDF.  In particular, MMDF must know the location

      and format of host tables for all  channels,  and  the

      channel  must  speak  a special protocol.  This allows

      MMDF to do additional verification (such as  verifying

      host names) at submission time.


           MMDF strictly separates the submission and deliv-

      ery phases.  Although _s_e_n_d_m_a_i_l has the concept of each

      of these stages, they are integrated into one program,

      whereas in MMDF they are split into two programs.


   44..33..  MMeessssaaggee PPrroocceessssiinngg MMoodduullee


           The Message Processing Module (MPM) discussed  by

      Postel  [Postel79b]  matches _s_e_n_d_m_a_i_l closely in terms

      of its basic architecture.  However,  like  MMDF,  the

      MPM includes the network interface software as part of

      its domain.
____________________
   4Dynamic configuration tables are currently being consid-
ered  for MMDF; allowing the installer to select either com-
piled or dynamic tables.
   5The MMDF equivalent of a _s_e_n_d_m_a_i_l "mailer."












SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr             SSMMMM::99--2255


           MPM also  postulates  a  duplex  channel  to  the

      receiver, as does MMDF, thus allowing simpler handling

      of errors by the mailer than is possible in  _s_e_n_d_m_a_i_l.

      When  a message queued by _s_e_n_d_m_a_i_l is sent, any errors

      must be returned to the sender by the  mailer  itself.

      Both  MPM  and  MMDF  mailers  can return an immediate

      error response, and a single error processor can  cre-

      ate an appropriate response.


           MPM  prefers  passing the message as a structured

      object, with type-length-value tuples6.  Such  a  con-

      vention  requires  a much higher degree of cooperation

      between mailers than is  required  by  _s_e_n_d_m_a_i_l.   MPM

      also  assumes  a universally agreed upon internet name

      space (with each address in the form  of  a  net-host-

      user tuple), which _s_e_n_d_m_a_i_l does not.


55..  EEVVAALLUUAATTIIOONNSS AANNDD FFUUTTUURREE PPLLAANNSS


        _S_e_n_d_m_a_i_l  is  designed  to  work in a nonhomogeneous

   environment.  Every attempt is  made  to  avoid  imposing

   unnecessary  constraints on the underlying mailers.  This

   goal has driven much of the design.   One  of  the  major

   problems has been the lack of a uniform address space, as

   postulated in [Postel79a] and [Postel79b].



____________________
   6This is similar to the NBS standard.












SSMMMM::99--2266             SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr


        A nonuniform address space implies that a path  will

   be specified in all addresses, either explicitly (as part

   of the address) or implicitly (as with implied forwarding

   to gateways).  This restriction has the unpleasant effect

   of making replying  to  messages  exceedingly  difficult,

   since  there is no one "address" for any person, but only

   a way to get there from wherever you are.


        Interfacing to mail programs that were not initially

   intended  to  be  applied  in an internet environment has

   been amazingly successful, and has reduced the job  to  a

   manageable task.


        _S_e_n_d_m_a_i_l  has  knowledge of a few difficult environ-

   ments built in.  It generates ARPANET FTP/SMTP compatible

   error  messages (prepended with three-digit numbers [Nei-

   gus73, Postel74, Postel82]) as necessary, optionally gen-

   erates  UNIX-style  "From" lines on the front of messages

   for some mailers, and knows how to parse the  same  lines

   on  input.  Also, error handling has an option customized

   for BerkNet.


        The decision to avoid doing  any  type  of  delivery

   where possible (even, or perhaps especially, local deliv-

   ery) has turned out to be a good idea.  Even  with  local

   delivery,  there  are issues of the location of the mail-

   box, the format of  the  mailbox,  the  locking  protocol

   used, etc., that are best decided by other programs.  One











SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr             SSMMMM::99--2277


   surprisingly major annoyance in many internet mailers  is

   that  the  location and format of local mail is built in.

   The feeling seems to be that local mail is so common that

   it  should be efficient.  This feeling is not born out by

   our experience; on the contrary, the location and  format

   of  mailboxes seems to vary widely from system to system.


        The ability to automatically generate a response  to

   incoming  mail  (by  forwarding  mail to a program) seems

   useful ("I am on vacation until  late  August....")   but

   can  create problems such as forwarding loops (two people

   on vacation whose programs send notes back and forth, for

   instance) if these programs are not well written.  A pro-

   gram could be written to do standard tasks correctly, but

   this would solve the general case.


        It might be desirable to implement some form of load

   limiting.  I am unaware of any mail system that addresses

   this  problem,  nor am I aware of any reasonable solution

   at this time.


        The  configuration  file  is  currently  practically

   inscrutable;  considerable  convenience could be realized

   with a higher-level format.


        It seems clear that common protocols will be  chang-

   ing  soon  to accommodate changing requirements and envi-

   ronments.  These changes will  include  modifications  to

   the  message header (e.g., [NBS80]) or to the body of the










SSMMMM::99--2288             SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr


   message itself (such as  for  multimedia  messages  [Pos-

   tel80]).   Experience indicates that these changes should

   be relatively trivial to integrate into the existing sys-

   tem.


        In tightly coupled environments, it would be nice to

   have a name server such  as  Grapvine  [Birrell82]  inte-

   grated  into  the  mail  system.  This would allow a site

   such as "Berkeley" to appear as  a  single  host,  rather

   than  as a collection of hosts, and would allow people to

   move  transparently  among  machines  without  having  to

   change their addresses.  Such a facility would require an

   automatically updated database and some method of resolv-

   ing  conflicts.   Ideally  this  would  be effective even

   without all hosts being under a single management.   How-

   ever,  it  is  not  clear  whether this feature should be

   integrated into the aliasing facility or should  be  con-

   sidered  a "value added" feature outside _s_e_n_d_m_a_i_l itself.


        As a more interesting case, the  CSNET  name  server

   [Solomon81]  provides an facility that goes beyond a sin-

   gle tightly-coupled environment.  Such a  facility  would

   normally exist outside of _s_e_n_d_m_a_i_l however.


AACCKKNNOOWWLLEEDDGGEEMMEENNTTSS


     Thanks  are due to Kurt Shoens for his continual cheer-

ful assistance and good advice, Bill Joy for pointing me  in

the  correct  direction (over and over), and Mark Horton for










SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr             SSMMMM::99--2299


more advice, prodding, and many of the good ideas.  Kurt and

Eric  Schmidt  are to be credited for using _d_e_l_i_v_e_r_m_a_i_l as a

server for their programs (_M_a_i_l  and  BerkNet  respectively)

before any sane person should have, and making the necessary

modifications promptly and happily.  Eric gave me  consider-

able advice about the perils of network software which saved

me an unknown amount of work and grief.  Mark did the origi-

nal implementation of the DBM version of aliasing, installed

the VFORK code, wrote the current version of _r_m_a_i_l, and  was

the  person  who  really  convinced  me to put the work into

_d_e_l_i_v_e_r_m_a_i_l to turn it into _s_e_n_d_m_a_i_l.  Kurt  deserves  acco-

lades  for  using  _s_e_n_d_m_a_i_l when I was myself afraid to take

the risk; how a person can continue to be so enthusiastic in

the face of so much bitter reality is beyond me.


     Kurt,  Mark,  Kirk  McKusick,  Marvin Solomon, and many

others have reviewed this paper, giving considerable  useful

advice.


     Special  thanks  are  reserved  for Mike Stonebraker at

Berkeley and Bob Epstein at Britton-Lee, who both  knowingly

allowed  me to put so much work into this project when there

were so many other things I really should have been  working

on.
























                         REFERENCES


[Birrell82]    Birrell,  A.  D.,  Levin, R., Needham, R. M.,

               and Schroeder, M. D., "Grapevine: An Exercise

               in  Distributed  Computing."  In _C_o_m_m_. _A_._C_._M_.

               _2_5_, 4, April 82.


[Borden79]     Borden, S., Gaines, R. S.,  and  Shapiro,  N.

               Z.,  _T_h_e  _M_H  _M_e_s_s_a_g_e _H_a_n_d_l_i_n_g _S_y_s_t_e_m_: _U_s_e_r_s_'

               _M_a_n_u_a_l_.    R-2367-PAF.    Rand   Corporation.

               October 1979.


[Crocker77a]   Crocker, D. H., Vittal, J. J., Pogran, K. T.,

               and Henderson, D. A. Jr.,  _S_t_a_n_d_a_r_d  _f_o_r  _t_h_e

               _F_o_r_m_a_t  _o_f  _A_R_P_A  _N_e_t_w_o_r_k _T_e_x_t _M_e_s_s_a_g_e_s_.  RFC

               733, NIC 41952.   In  [Feinler78].   November

               1977.


[Crocker77b]   Crocker,  D.  H.,  _F_r_a_m_e_w_o_r_k _a_n_d _F_u_n_c_t_i_o_n_s _o_f

               _t_h_e _M_S _P_e_r_s_o_n_a_l _M_e_s_s_a_g_e _S_y_s_t_e_m_.  R-2134-ARPA,

               Rand  Corporation,  Santa Monica, California.

               1977.


[Crocker79]    Crocker, D. H., Szurkowski, E. S.,  and  Far-

               ber, D. J., _A_n _I_n_t_e_r_n_e_t_w_o_r_k _M_e_m_o _D_i_s_t_r_i_b_u_t_i_o_n

               _F_a_c_i_l_i_t_y _-_-  _M_M_D_F_.   6th  Data  Communication

               Symposium, Asilomar.  November 1979.




SSMMMM::99--3300             SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr







SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr             SSMMMM::99--3311


[Crocker82]    Crocker,  D.  H.,  _S_t_a_n_d_a_r_d _f_o_r _t_h_e _F_o_r_m_a_t _o_f

               _A_r_p_a _I_n_t_e_r_n_e_t _T_e_x_t _M_e_s_s_a_g_e_s_.  RFC 822.   Net-

               work  Information  Center, SRI International,

               Menlo Park, California.  August 1982.


[Metcalfe76]   Metcalfe, R., and Boggs, D., "Ethernet:  Dis-

               tributed  Packet Switching for Local Computer

               Networks", _C_o_m_m_u_n_i_c_a_t_i_o_n_s _o_f _t_h_e _A_C_M  _1_9_,  7.

               July 1976.


[Feinler78]    Feinler,  E., and Postel, J.  (eds.), _A_R_P_A_N_E_T

               _P_r_o_t_o_c_o_l _H_a_n_d_b_o_o_k_.  NIC 7104, Network  Infor-

               mation Center, SRI International, Menlo Park,

               California.  1978.


[NBS80]        National Bureau of  Standards,  _S_p_e_c_i_f_i_c_a_t_i_o_n

               _o_f  _a  _D_r_a_f_t _M_e_s_s_a_g_e _F_o_r_m_a_t _S_t_a_n_d_a_r_d_.  Report

               No. ICST/CBOS 80-2.  October 1980.


[Neigus73]     Neigus, N., _F_i_l_e _T_r_a_n_s_f_e_r  _P_r_o_t_o_c_o_l  _f_o_r  _t_h_e

               _A_R_P_A _N_e_t_w_o_r_k_.  RFC 542, NIC 17759.  In [Fein-

               ler78].  August, 1973.


[Nowitz78a]    Nowitz, D. A., and Lesk,  M.  E.,  _A  _D_i_a_l_-_U_p

               _N_e_t_w_o_r_k  _o_f _U_N_I_X _S_y_s_t_e_m_s_.  Bell Laboratories.

               In UNIX Programmer's Manual, Seventh Edition,

               Volume 2.  August, 1978.


[Nowitz78b]    Nowitz,  D.  A., _U_u_c_p _I_m_p_l_e_m_e_n_t_a_t_i_o_n _D_e_s_c_r_i_p_-

               _t_i_o_n_.    Bell    Laboratories.     In    UNIX










SSMMMM::99--3322             SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr


               Programmer's  Manual, Seventh Edition, Volume

               2.  October, 1978.


[Postel74]     Postel, J., and Neigus, N., Revised FTP Reply

               Codes.   RFC 640, NIC 30843.  In [Feinler78].

               June, 1974.


[Postel77]     Postel, J., _M_a_i_l _P_r_o_t_o_c_o_l_.   NIC  29588.   In

               [Feinler78].  November 1977.


[Postel79a]    Postel,  J.,  _I_n_t_e_r_n_e_t _M_e_s_s_a_g_e _P_r_o_t_o_c_o_l_.  RFC

               753, IEN 85.  Network Information Center, SRI

               International, Menlo Park, California.  March

               1979.


[Postel79b]    Postel, J. B., _A_n _I_n_t_e_r_n_e_t_w_o_r_k _M_e_s_s_a_g_e _S_t_r_u_c_-

               _t_u_r_e_.   In _P_r_o_c_e_e_d_i_n_g_s _o_f _t_h_e _S_i_x_t_h _D_a_t_a _C_o_m_-

               _m_u_n_i_c_a_t_i_o_n_s  _S_y_m_p_o_s_i_u_m_,  IEEE.    New   York.

               November 1979.


[Postel80]     Postel, J. B., _A _S_t_r_u_c_t_u_r_e_d _F_o_r_m_a_t _f_o_r _T_r_a_n_s_-

               _m_i_s_s_i_o_n _o_f _M_u_l_t_i_-_M_e_d_i_a _D_o_c_u_m_e_n_t_s_.   RFC  767.

               Network   Information  Center,  SRI  Interna-

               tional, Menlo Park, California.  August 1980.


[Postel82]     Postel, J. B., _S_i_m_p_l_e _M_a_i_l _T_r_a_n_s_f_e_r _P_r_o_t_o_c_o_l_.

               RFC821 (obsoleting RFC788).  Network Informa-

               tion  Center,  SRI International, Menlo Park,

               California.  August 1982.












SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr             SSMMMM::99--3333


[Schmidt79]    Schmidt, E., _A_n _I_n_t_r_o_d_u_c_t_i_o_n _t_o _t_h_e  _B_e_r_k_e_l_e_y

               _N_e_t_w_o_r_k_.   University of California, Berkeley

               California.  1979.


[Shoens79]     Shoens, K., _M_a_i_l _R_e_f_e_r_e_n_c_e  _M_a_n_u_a_l_.   Univer-

               sity  of  California, Berkeley.  In UNIX Pro-

               grammer's Manual, Seventh Edition, Volume 2C.

               December 1979.


[Sluizer81]    Sluizer, S., and Postel, J. B., _M_a_i_l _T_r_a_n_s_f_e_r

               _P_r_o_t_o_c_o_l_.  RFC 780.  Network Information Cen-

               ter,  SRI International, Menlo Park, Califor-

               nia.  May 1981.


[Solomon81]    Solomon, M., Landweber,  L.,  and  Neuhengen,

               D.,  "The  Design  of the CSNET Name Server."

               CS-DN-2, University  of  Wisconsin,  Madison.

               November 1981.


[Su82]         Su,  Zaw-Sing,  and  Postel,  Jon, _T_h_e _D_o_m_a_i_n

               _N_a_m_i_n_g _C_o_n_v_e_n_t_i_o_n _f_o_r _I_n_t_e_r_n_e_t _U_s_e_r  _A_p_p_l_i_c_a_-

               _t_i_o_n_s_.   RFC819.  Network Information Center,

               SRI International,  Menlo  Park,  California.

               August 1982.


[UNIX83]       _T_h_e  _U_N_I_X  _P_r_o_g_r_a_m_m_e_r_'_s  _M_a_n_u_a_l_, _S_e_v_e_n_t_h _E_d_i_-

               _t_i_o_n_, Virtual VAX-11 Version, Volume 1.  Bell

               Laboratories,  modified  by the University of

               California,  Berkeley,  California.    March,











SSMMMM::99--3344             SSEENNDDMMAAIILL ---- AAnn IInntteerrnneettwwoorrkk MMaaiill RRoouutteerr


               1983.


























































