New SRS patch

[Christoph Garst]

Hi,

i've made a new SRS patch for debian postfix-2.7.0.
It is based on Sheveks patch for postfix-2.1.4 but has many
improvements:

Works with all address classes (not limited to local domain).
Doesn't depend on .forward-files or alias_maps.
SMTPD accepts valid srs addresses without further configuration.
Sender gets rewritten only when necessary:
- rewrite after queuing (after separation to on- and off-site
destinations)
- rewrite only if sender and recipient are off-site and host is not a
relay
for recipient
Alias domain can be chosen automatically from original
on-site-recipient.

Patch is available at http://www.codefrickler.de/srs

I decided NOT to write a milter/filter plugin for the following
reasons:
Forward rewriting has to be done AFTER queuing, in order to
distinguish between on-site and off-site recipients. One could have
used a
filter for that, but an external filter doesn't have direct access to
postfixs lookup tables nor the original recipient nor the queue id for
logging purposes. So it's more complicated to get it up and running,
especially if you have many domains.

Chris

[Wietse Venema]

Post by Christoph Garst
I decided NOT to write a milter/filter plugin for the following
Forward rewriting has to be done AFTER queuing, in order to
distinguish between on-site and off-site recipients.

I suppose that is one reason why some sites use separate MTAs for
their inbound and outbound mail. The inbound MTA strips SRS from
local recipients, and the outbound MTA adds SRS to local senders.

This model can be implemented with Postfix using the existing
content-manipulation interfaces.

Wietse

[Wietse Venema]

Post by Wietse Venema

Post by Christoph Garst
I decided NOT to write a milter/filter plugin for the following
Forward rewriting has to be done AFTER queuing, in order to
distinguish between on-site and off-site recipients.

I suppose that is one reason why some sites use separate MTAs for
their inbound and outbound mail. The inbound MTA strips SRS from
local recipients, and the outbound MTA adds SRS to local senders.

I looked at the patch, and the approach taken there is limited to
MTAs that perform final delivery (the decision to un-SRS the recipient
is hard-coded for final delivery only). That is a major limitation.

This does not work on a mail gateway configuration, where Postfix
is a transit MTA that handles mail on behalf of one or more internal
MTAs. Transit MTAs don't perform final delivery - that is done by
the internal MTAs (Postfix-in-front-of-Exchange being one example).

On the other hand, when mail authentication is designed so that it
is not limited to final-delivery MTAs, then the solution becomes
scalable.

I would rather provide hooks that allow Postfix to co-exist with
mail authentication systems, than hard-code those systems into the
core MTA. By the time the hard-coded version finally ends up in
binary distributions (years!), the code is already obsolete.

Wietse

[Christoph Garst]

Post by Wietse Venema
I suppose that is one reason why some sites use separate MTAs for
their inbound and outbound mail. The inbound MTA strips SRS from
local recipients, and the outbound MTA adds SRS to local senders

Why do I want to add SRS to local senders?
I need SRS for non-local senders, primariliy to forward local addresses
to external domains.

Post by Wietse Venema
This model can be implemented with Postfix using the existing
content-manipulation interfaces.

Ok, adding SRS can be done by a filter plugin (getting the alias domain
out of the original recipient is not necessarily a must-have feature),
but then the filter needs to know which domains are local.

Post by Wietse Venema
I looked at the patch, and the approach taken there is limited to
MTAs that perform final delivery (the decision to un-SRS the recipient
is hard-coded for final delivery only). That is a major limitation.
This does not work on a mail gateway configuration, where Postfix
is a transit MTA that handles mail on behalf of one or more internal
MTAs. Transit MTAs don't perform final delivery - that is done by
the internal MTAs (Postfix-in-front-of-Exchange being one example).

For incoming mail I don't see the problem to un-SRS the recipient if we
have the appropriate secret. In smtpd there's only a check if the
SRS-address is valid, rewriting itself is done in cleanup together with
canonical address mapping.

I doubt to use SMTP-milter since it can't make modifications to the
envelope. Or does this limitation only apply if additionally using a
content filter?

[Wietse Venema]

Post by Christoph Garst
I doubt to use SMTP-milter since it can't make modifications to the
envelope.

SMFIR_CHGFROM and SMFIR_ADDRCPT_PAR support exists since Postfix
2.6, four stable releases ago. Support for SMFIR_ADDRCPT/SMFIR_DELRCPT
exists even longer.

Wietse

[Christoph Garst]

Post by Wietse Venema

Post by Christoph Garst
I doubt to use SMTP-milter since it can't make modifications to the
envelope.

SMFIR_CHGFROM and SMFIR_ADDRCPT_PAR support exists since Postfix
2.6, four stable releases ago. Support for SMFIR_ADDRCPT/SMFIR_DELRCPT
exists even longer.

From MILTER_README: When you use the before-queue content filter for
incoming SMTP mail (see SMTPD_PROXY_README), Milter applications have
access only to the SMTP command information; they have no access to the
message header or body, and cannot make modifications to the message or
to the envelope.

Is this obsolete?

[Wietse Venema]

Post by Christoph Garst

Post by Wietse Venema

Post by Christoph Garst
I doubt to use SMTP-milter since it can't make modifications to the
envelope.

SMFIR_CHGFROM and SMFIR_ADDRCPT_PAR support exists since Postfix
2.6, four stable releases ago. Support for SMFIR_ADDRCPT/SMFIR_DELRCPT
exists even longer.

From MILTER_README: When you use the before-queue content filter for
incoming SMTP mail (see SMTPD_PROXY_README), Milter applications have
access only to the SMTP command information; they have no access to the
message header or body, and cannot make modifications to the message or
to the envelope.
Is this obsolete?

It is 100% correct. To change the message or envelope you'd have
to configure the Milter BEHIND the SMTPD_PROXY filter.

Wietse

[Wietse Venema]

Post by Christoph Garst
Hi,
i've made a new SRS patch for debian postfix-2.7.0.
It is based on Sheveks patch for postfix-2.1.4 but has many

I think I have found one architectural problem with this implementation.

It SRS-ifies the envelope sender BEFORE giving it to a delivery
agent. The problem with this is that the delivery agent will report
delivery status notifications to the SRS-ified envelope sender
address.

This can be avoided by SRS-ifying the sender in the delivery agent
while it encapsulates the message into SMTP. SRS is really a
transformation that is tied to the SMTP protocol.

This could be addressed by making smtp_generic_maps more general,
such that there are separate variants for envelope sender, envelope
recipient, and likewise the message header addresses (perhaps with
names like smtp_sender_envelope_generic_maps, etc.)

Wietse

[Wietse Venema]

Post by Wietse Venema

Post by Christoph Garst
Hi,
i've made a new SRS patch for debian postfix-2.7.0.
It is based on Sheveks patch for postfix-2.1.4 but has many

I think I have found one architectural problem with this implementation.
It SRS-ifies the envelope sender BEFORE giving it to a delivery
agent. The problem with this is that the delivery agent will report
delivery status notifications to the SRS-ified envelope sender
address.

On closer investigation, it seems that the patch unconditionally
un-SRSes the envelope recipient (if it can) as it enqueues mail.
Thus, the DSN message will be delivered instead.

However, the SRS-ified envelope sender address would still show up
in the To: header of the DSN message.

Wietse

[Viktor Dukhovni]

Post by Wietse Venema
However, the SRS-ified envelope sender address would still show up
in the To: header of the DSN message.

This is a problem, since it breaks VERP in bounce handlers, so indeed
the patch should avoid rewriting the sender too early.

Which one to the question of what to do with inbound bounces from
the downstream site which arrive to the SRS rewritten address, does
the SRS patch rewrite the headers as well as the envelope?

--
Viktor.

[Christoph Garst]

SRS is really a transformation that is tied to the SMTP protocol.

I agree, SMTP would be a better place to transform the sender.

This could be addressed by making smtp_generic_maps more general,
such that there are separate variants for envelope sender, envelope
recipient, and likewise the message header addresses (perhaps with
names like smtp_sender_envelope_generic_maps, etc.)

Extending smtp_generic_maps is nice, but won't help in this situation.
A lookup table isn't sufficient to do the rewriting, we need a function
like postfix_srs_forward (in analogy to verp_sender).

I don't quite understand the DSN problem. If the envelope recipient is
a srs address, the message is most likely a bounce. Are there any cases
where someone is interested in DSNs of bounces?

Does the SRS patch rewrite the headers as well as the envelope?

No, headers aren't touched in SRS.

[Wietse Venema]

Post by Christoph Garst

SRS is really a transformation that is tied to the SMTP protocol.

I agree, SMTP would be a better place to transform the sender.

This could be addressed by making smtp_generic_maps more general,
such that there are separate variants for envelope sender, envelope
recipient, and likewise the message header addresses (perhaps with
names like smtp_sender_envelope_generic_maps, etc.)

Extending smtp_generic_maps is nice, but won't help in this situation.
A lookup table isn't sufficient to do the rewriting, we need a function
like postfix_srs_forward (in analogy to verp_sender).

Postfix lookup tables are not limited to static files. One
implementation of SRS for Sendmail uses socketmaps, and I just added
Sendmail-style socketmap support to Postfix.

If the smtp_generic_maps feature were to be broken up into
smtp_{envelope,header}_{sender,recipient}_generic_maps then SRS
could plug into smtp_envelope_sender_generic_maps.

Post by Christoph Garst
I don't quite understand the DSN problem. If the envelope recipient is
a srs address, the message is most likely a bounce. Are there any cases
where someone is interested in DSNs of bounces?

Per your patch, the queue manager gives an SRS-ified envelope sender
to Postfix delivery agents. When a delivery agent can't deliver
mail, or when the sender has requested a DSN for other reasons,
then Postfix will send a DSN with the SRS-ified sender address in
the To: header. I suppose that is a feature of SRS, because a
bounce from a remote system would also look that way.

I was concerned that Postfix would not know how to deliver the
bounce, but it turns out that your patch will attempt to strip SRS
from all recipients as mail is queued, not as mail arrives via SMTP.

Wietse